summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2014-05-26Small bugfixes to the SearchImplHEADmasterMaurice Laveaux1-19/+28
* Convers ids to longs instead of ints * input and output fiel correctly named * RateLimit is handled better. Signed-off-by: Maurice Laveaux <maurice@localhost.localdomain>
2014-05-26Added a resume option.Maurice Laveaux1-10/+13
* Resumes searching of tweets.
2014-05-26Made it possible to set the output filenameMaurice Laveaux2-4/+15
* --config <filename> option added. * Configuration is now a singleton.
2014-05-26Improved the requesterMaurice Laveaux2-13/+28
* Reads a gzip stream. * Uses the http request date for interval measurement.
2014-05-26Fixed small bug.Maurice Laveaux1-1/+1
2014-05-22Correctness fixesPeter Wu3-41/+46
* Throw interrupted exception when interrupted instead of swallowing it, otherwise shutdown will take a longer time. * Decrease number of tweets that must be requested after confirming that the request was successful. * Style: convert to foreach; white-space. * Properly terminate if stop is requested. * Make executeRequest take care of re-queing requests if needed.
2014-05-22Properly close files after load/savePeter Wu1-16/+14
2014-05-22Rename getResp to getJsonPeter Wu4-16/+16
2014-05-22Removed unused port.Maurice Laveaux1-2/+0
* I dun goofed.
2014-05-22Updated the gitignore file.Maurice Laveaux1-0/+1
* ignore the queue_cache
2014-05-22Added saving and loaded the search cache.Maurice Laveaux1-12/+56
* TODO: threads don't exit safe, so cache is not valid.
2014-05-22Removed unused source filesMaurice Laveaux47-5375/+0
* Removed the shitty examples.
2014-05-22Implemented basic search functionality.Maurice Laveaux2-13/+372
* Usage, search <screen_name> <amount_tweets>.
2014-05-22Fixed: Twitter API didn't throw RateLimitException.Maurice Laveaux1-2/+2
2014-05-22Removed all JSON references.Maurice Laveaux4-9/+2
2014-05-22Refactored JSON to GSON's implementation.Maurice Laveaux7-85/+97
* Changed JSONException to JsonParseException.
2014-05-22Updated the Requesters to throw RateLimitException.Maurice Laveaux5-17/+54
* The exception contains the reset timestamp in unix time.
2014-05-22Changed Main so it handles Response objects.Maurice Laveaux1-15/+12
2014-05-22Added the Search interface.Maurice Laveaux1-0/+49
2014-05-22Changed Request response to a new Response class.Maurice Laveaux6-64/+86
* Response contains the JsonElement, tickrate, reset and leftover data. * Removed getJsonRelaxed, because it was not used elsewhere.
2014-05-08Delete junkPeter Wu6-548/+0
2014-05-08CompressableDataWriter: fix corruption for cfilePeter Wu1-0/+4
2014-05-07Compare against exactly class, not just the compatible classPeter Wu1-1/+1
CompressableDataWriter extends DataWriter. We require classes to be an exact match. This fixes disabling file, and (re-)enabling cfile even if that target is not changed.
2014-05-07Fix null ptr when close/commitPeter Wu1-1/+8
commit closes everything via safeClose, so ensure that tweetGenerated re-opens it if necessary. Fixes: ƒ commit € Exception in thread "Thread-7" java.lang.NullPointerException at io.DataWriter.writeObject(DataWriter.java:137) at io.DataWriter.tweetGenerated(DataWriter.java:85) at provider.CompositeResultListener.tweetGenerated(CompositeResultListener.java:51) at io.StreamImpl$Poller.processObject(StreamImpl.java:255) at io.StreamImpl$Poller.run(StreamImpl.java:245) at java.lang.Thread.run(Thread.java:744)
2014-05-07Display user for tweet, strip whitespace around keywordPeter Wu2-1/+5
2014-05-07Add flush commandPeter Wu3-2/+39
2014-05-07Add timing state informationPeter Wu2-0/+22
2014-05-07Add command to repeat last command.Peter Wu1-1/+10
2014-05-07Shutdown hook for safely closing filesPeter Wu1-0/+8
2014-05-07F it, get rid of junkPeter Wu1-15/+0
2014-05-07Prevent enabling both cfile and filePeter Wu2-3/+15
2014-05-07Get rid of separate profilesPeter Wu8-79/+12
These are always available from the tweets themselves...
2014-05-07Allow commit to disconnect for empty keywords, detect dead connectionPeter Wu2-6/+21
2014-05-07Set read timeout for streamingPeter Wu1-0/+3
2014-05-07Fix deadlock in Poller if worker thread stops/diesPeter Wu1-0/+10
2014-05-07StreamImpl: make (un)watchKeywords support a listPeter Wu1-4/+20
2014-05-07Use IOUtils.closeQuietly for less verbosityPeter Wu1-13/+3
2014-05-03TweetShell: Safely close files to prevent file corruptionPeter Wu1-0/+2
2014-05-03DataWriter: first read, then open for writingPeter Wu1-2/+3
Prevents occasional "Unexpected end of ZLIB input stream" errors.
2014-05-03DataWriter: better detect errors in reading dataPeter Wu1-6/+13
Now print the faulty line and line number. Using BufferedReader instead of Scanner also gave me a hint of the error, so let's stick to it.
2014-05-03Rename config option, default to converting uncompressed filesPeter Wu1-2/+2
2014-05-03CompressableDataWriter: use separate .gz-suffixed filePeter Wu1-91/+72
Do not use the same name for compressed and uncompressed files, it is confusing and non-standard. Now it'll use a ".gz"-suffixed file, writing all changes from the uncompressed file if requested.
2014-05-03Display stats when converting an uncompressed filePeter Wu1-0/+18
2014-05-03More compression fixesPeter Wu1-9/+12
* Fix magic check (bytes are read as signed numbers...). * Assume that a file can be compressed if the file is too small. * Try to convert a file only if the gzip detection is correctly closed.
2014-05-02Accept multiple commands (for automation)Peter Wu2-8/+15
Instead of `./run.sh shell target +file`, and then manually running add and commit, you can now use: ./run.sh shell 'target +file' 'add samsung' commit Quoting is necessary because of the spaces.
2014-05-02Properly close files and stream on exitPeter Wu3-7/+32
2014-05-02Accept "force-gzip" option to force compressionPeter Wu1-2/+4
2014-05-02Fix compression move which actually lost some dataPeter Wu1-28/+21
rename tricks somehow failed, but apparently there is a reliable Files.move() operation. Add some more debugging messages.
2014-05-02DataWriter should not open files in constructorPeter Wu3-21/+47
Right now Store instances are created in the constructor of DataWriter. This causes issues, namely that Store instances are unable to use DataWriter in its constructor. The real-world implication is that convertUncompressed in CompressableDataWriter is unusable for the CompressedStore.
2014-05-02Reject commits without keywordsPeter Wu2-1/+16