summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)AuthorFilesLines
2014-05-07Add timing state informationPeter Wu2-0/+22
2014-05-07Add command to repeat last command.Peter Wu1-1/+10
2014-05-07Shutdown hook for safely closing filesPeter Wu1-0/+8
2014-05-07F it, get rid of junkPeter Wu1-15/+0
2014-05-07Prevent enabling both cfile and filePeter Wu2-3/+15
2014-05-07Get rid of separate profilesPeter Wu8-79/+12
These are always available from the tweets themselves...
2014-05-07Allow commit to disconnect for empty keywords, detect dead connectionPeter Wu2-6/+21
2014-05-07Set read timeout for streamingPeter Wu1-0/+3
2014-05-07Fix deadlock in Poller if worker thread stops/diesPeter Wu1-0/+10
2014-05-07StreamImpl: make (un)watchKeywords support a listPeter Wu1-4/+20
2014-05-07Use IOUtils.closeQuietly for less verbosityPeter Wu1-13/+3
2014-05-03TweetShell: Safely close files to prevent file corruptionPeter Wu1-0/+2
2014-05-03DataWriter: first read, then open for writingPeter Wu1-2/+3
Prevents occasional "Unexpected end of ZLIB input stream" errors.
2014-05-03DataWriter: better detect errors in reading dataPeter Wu1-6/+13
Now print the faulty line and line number. Using BufferedReader instead of Scanner also gave me a hint of the error, so let's stick to it.
2014-05-03Rename config option, default to converting uncompressed filesPeter Wu1-2/+2
2014-05-03CompressableDataWriter: use separate .gz-suffixed filePeter Wu1-91/+72
Do not use the same name for compressed and uncompressed files, it is confusing and non-standard. Now it'll use a ".gz"-suffixed file, writing all changes from the uncompressed file if requested.
2014-05-03Display stats when converting an uncompressed filePeter Wu1-0/+18
2014-05-03More compression fixesPeter Wu1-9/+12
* Fix magic check (bytes are read as signed numbers...). * Assume that a file can be compressed if the file is too small. * Try to convert a file only if the gzip detection is correctly closed.
2014-05-02Accept multiple commands (for automation)Peter Wu2-8/+15
Instead of `./run.sh shell target +file`, and then manually running add and commit, you can now use: ./run.sh shell 'target +file' 'add samsung' commit Quoting is necessary because of the spaces.
2014-05-02Properly close files and stream on exitPeter Wu3-7/+32
2014-05-02Accept "force-gzip" option to force compressionPeter Wu1-2/+4
2014-05-02Fix compression move which actually lost some dataPeter Wu1-28/+21
rename tricks somehow failed, but apparently there is a reliable Files.move() operation. Add some more debugging messages.
2014-05-02DataWriter should not open files in constructorPeter Wu3-21/+47
Right now Store instances are created in the constructor of DataWriter. This causes issues, namely that Store instances are unable to use DataWriter in its constructor. The real-world implication is that convertUncompressed in CompressableDataWriter is unusable for the CompressedStore.
2014-05-02Reject commits without keywordsPeter Wu2-1/+16
2014-05-02Fix typo in DataWriterPeter Wu1-1/+1
2014-05-02Abort if file cannot be read, don't leak on errorPeter Wu1-11/+17
2014-05-02CompressableDataWriter: Implement conversion from uncompressedPeter Wu1-1/+64
2014-05-02Support compressed filesPeter Wu2-2/+99
2014-05-02Adjust target command help message, print available targetsPeter Wu1-7/+10
2014-05-02DataWriter: don't leak input fdPeter Wu1-1/+10
2014-05-02Refactor DataWriter once more to group a filePeter Wu1-37/+79
The input and output file are closely related, do not burden others with tracking which output and input need to be matched.
2014-05-02Make DataWriter more genericPeter Wu1-11/+40
Prepare for compressed file formats, this requires that the data writer can handle compressed files. In theory an implementation can even write stuff over the network instead of a file.
2014-05-02Track number of unique users for status reportingPeter Wu2-1/+27
2014-05-02Don't ignore IOException stupid!Peter Wu2-39/+33
Files may not be accessible if they are read-only, or if the directory is read-only, or ... whatever. So, let the caller handle IOException in DataWriter. Return a set if it can be read. Do some misc cleanups (remove unused imports, fix docs, remove obsolete TODOs).
2014-05-02Basic stats for tweetsPeter Wu2-0/+39
2014-05-02Fix enabled targets print, allow multiple args, improve helpPeter Wu1-4/+10
2014-05-02Persist a single set of targets for all streamsPeter Wu1-31/+32
* Split ResultListener and ExceptionListener implementation, the methods turns out not to be closely related. * Extract CompositeResultListener from Stream, making TweetShell own it, this simplifies code and allows you to have targets even if no stream is instantiated.
2014-05-02Allow multiple targets to be enabled/disabledPeter Wu3-28/+273
2014-05-02Show hint if no keywords are addedPeter Wu1-1/+5
2014-05-02Add "?" alias for "help"Peter Wu1-1/+5
2014-05-02Avoid hang during closePeter Wu1-4/+29
Apparently the socket operations are blocking in a way such that even interrupt() cannot stop it. Close the socket (which will generate an IOException masked as JSONException) to immediately close the socket. Also do not leak a socket if an error occurred in the Worker constructor (when a Stream connection is set up).
2014-05-01Add command to print active keywordsPeter Wu3-0/+47
2014-05-01Don't leak connectionsPeter Wu2-34/+42
2014-05-01Prettify prompt, fix target paramPeter Wu1-2/+7
2014-05-01StreamHandler now outputs the tweet detailsPeter Wu1-2/+17
2014-05-01New thread for notifying of new objectsPeter Wu1-13/+95
* Use volatile boolean instead of AtomicBoolean. There will only be one one writer. * Store received objects in a queue instead of printing them. * Rename thread to clarify its I/O purpose. * Create new Thread for notifying observers of new tweets and users.
2014-05-01Get rid of unused and broken poll functionPeter Wu3-18/+0
2014-05-01Do not destroy the old file...Peter Wu1-6/+6
2014-05-01Merge branch 'master' of git.lekensteyn.nl:tue/2IOC0-DBL/TwitterDataAnalyticsMaurice Laveaux1-5/+11
2014-05-01Moved setting the default values.Maurice Laveaux2-5/+11
* Moved from TweetShell to Configuration.