Age | Commit message (Collapse) | Author | Files | Lines | |
---|---|---|---|---|---|
2014-05-22 | Refactored JSON to GSON's implementation. | Maurice Laveaux | 2 | -36/+46 | |
* Changed JSONException to JsonParseException. | |||||
2014-05-22 | Updated the Requesters to throw RateLimitException. | Maurice Laveaux | 5 | -17/+54 | |
* The exception contains the reset timestamp in unix time. | |||||
2014-05-22 | Changed Request response to a new Response class. | Maurice Laveaux | 5 | -62/+83 | |
* Response contains the JsonElement, tickrate, reset and leftover data. * Removed getJsonRelaxed, because it was not used elsewhere. | |||||
2014-05-08 | CompressableDataWriter: fix corruption for cfile | Peter Wu | 1 | -0/+4 | |
2014-05-07 | Fix null ptr when close/commit | Peter Wu | 1 | -1/+8 | |
commit closes everything via safeClose, so ensure that tweetGenerated re-opens it if necessary. Fixes: ƒ commit € Exception in thread "Thread-7" java.lang.NullPointerException at io.DataWriter.writeObject(DataWriter.java:137) at io.DataWriter.tweetGenerated(DataWriter.java:85) at provider.CompositeResultListener.tweetGenerated(CompositeResultListener.java:51) at io.StreamImpl$Poller.processObject(StreamImpl.java:255) at io.StreamImpl$Poller.run(StreamImpl.java:245) at java.lang.Thread.run(Thread.java:744) | |||||
2014-05-07 | Display user for tweet, strip whitespace around keyword | Peter Wu | 1 | -0/+1 | |
2014-05-07 | Add flush command | Peter Wu | 1 | -1/+13 | |
2014-05-07 | Get rid of separate profiles | Peter Wu | 3 | -42/+7 | |
These are always available from the tweets themselves... | |||||
2014-05-07 | Allow commit to disconnect for empty keywords, detect dead connection | Peter Wu | 1 | -6/+6 | |
2014-05-07 | Set read timeout for streaming | Peter Wu | 1 | -0/+3 | |
2014-05-07 | Fix deadlock in Poller if worker thread stops/dies | Peter Wu | 1 | -0/+10 | |
2014-05-07 | StreamImpl: make (un)watchKeywords support a list | Peter Wu | 1 | -4/+20 | |
2014-05-07 | Use IOUtils.closeQuietly for less verbosity | Peter Wu | 1 | -13/+3 | |
2014-05-03 | DataWriter: first read, then open for writing | Peter Wu | 1 | -2/+3 | |
Prevents occasional "Unexpected end of ZLIB input stream" errors. | |||||
2014-05-03 | DataWriter: better detect errors in reading data | Peter Wu | 1 | -6/+13 | |
Now print the faulty line and line number. Using BufferedReader instead of Scanner also gave me a hint of the error, so let's stick to it. | |||||
2014-05-03 | CompressableDataWriter: use separate .gz-suffixed file | Peter Wu | 1 | -91/+72 | |
Do not use the same name for compressed and uncompressed files, it is confusing and non-standard. Now it'll use a ".gz"-suffixed file, writing all changes from the uncompressed file if requested. | |||||
2014-05-03 | Display stats when converting an uncompressed file | Peter Wu | 1 | -0/+18 | |
2014-05-03 | More compression fixes | Peter Wu | 1 | -9/+12 | |
* Fix magic check (bytes are read as signed numbers...). * Assume that a file can be compressed if the file is too small. * Try to convert a file only if the gzip detection is correctly closed. | |||||
2014-05-02 | Properly close files and stream on exit | Peter Wu | 1 | -1/+2 | |
2014-05-02 | Fix compression move which actually lost some data | Peter Wu | 1 | -28/+21 | |
rename tricks somehow failed, but apparently there is a reliable Files.move() operation. Add some more debugging messages. | |||||
2014-05-02 | DataWriter should not open files in constructor | Peter Wu | 2 | -21/+46 | |
Right now Store instances are created in the constructor of DataWriter. This causes issues, namely that Store instances are unable to use DataWriter in its constructor. The real-world implication is that convertUncompressed in CompressableDataWriter is unusable for the CompressedStore. | |||||
2014-05-02 | Reject commits without keywords | Peter Wu | 1 | -0/+4 | |
2014-05-02 | Fix typo in DataWriter | Peter Wu | 1 | -1/+1 | |
2014-05-02 | Abort if file cannot be read, don't leak on error | Peter Wu | 1 | -11/+17 | |
2014-05-02 | CompressableDataWriter: Implement conversion from uncompressed | Peter Wu | 1 | -1/+64 | |
2014-05-02 | Support compressed files | Peter Wu | 1 | -0/+88 | |
2014-05-02 | DataWriter: don't leak input fd | Peter Wu | 1 | -1/+10 | |
2014-05-02 | Refactor DataWriter once more to group a file | Peter Wu | 1 | -37/+79 | |
The input and output file are closely related, do not burden others with tracking which output and input need to be matched. | |||||
2014-05-02 | Make DataWriter more generic | Peter Wu | 1 | -11/+40 | |
Prepare for compressed file formats, this requires that the data writer can handle compressed files. In theory an implementation can even write stuff over the network instead of a file. | |||||
2014-05-02 | Don't ignore IOException stupid! | Peter Wu | 1 | -34/+23 | |
Files may not be accessible if they are read-only, or if the directory is read-only, or ... whatever. So, let the caller handle IOException in DataWriter. Return a set if it can be read. Do some misc cleanups (remove unused imports, fix docs, remove obsolete TODOs). | |||||
2014-05-02 | Avoid hang during close | Peter Wu | 1 | -4/+29 | |
Apparently the socket operations are blocking in a way such that even interrupt() cannot stop it. Close the socket (which will generate an IOException masked as JSONException) to immediately close the socket. Also do not leak a socket if an error occurred in the Worker constructor (when a Stream connection is set up). | |||||
2014-05-01 | Add command to print active keywords | Peter Wu | 1 | -0/+15 | |
2014-05-01 | Don't leak connections | Peter Wu | 2 | -34/+42 | |
2014-05-01 | New thread for notifying of new objects | Peter Wu | 1 | -13/+95 | |
* Use volatile boolean instead of AtomicBoolean. There will only be one one writer. * Store received objects in a queue instead of printing them. * Rename thread to clarify its I/O purpose. * Create new Thread for notifying observers of new tweets and users. | |||||
2014-05-01 | Get rid of unused and broken poll function | Peter Wu | 1 | -9/+0 | |
2014-05-01 | Merge branch 'master' of git.lekensteyn.nl:tue/2IOC0-DBL/TwitterDataAnalytics | Maurice Laveaux | 1 | -5/+11 | |
2014-05-01 | Added the target command with options: file, shell. | Maurice Laveaux | 2 | -4/+19 | |
* Implemented target file, which uses DataWriter as RequestListener which writes the tweets and profiles. * Added profiles-filename, tweets-filename to the config. | |||||
2014-05-01 | Close stream when stopping worker, English tweets only please... | Peter Wu | 1 | -5/+11 | |
2014-05-01 | Allow result and exception handler to be changed | Peter Wu | 1 | -7/+29 | |
2014-05-01 | Fix signing for OAuth POST messages, fix search param | Peter Wu | 3 | -6/+42 | |
2014-05-01 | Show more useful IO exception | Peter Wu | 1 | -1/+2 | |
2014-05-01 | [WIP] Initial hacky TweetShell | Peter Wu | 1 | -2/+26 | |
In TESTING! DefaultOAuthConsumer does not play nice with POST requests... need to fix that somehow. | |||||
2014-04-30 | Use wrapped gzipinputstream from twitter | Peter Wu | 1 | -2/+2 | |
2014-04-30 | StreamImpl: Read JSON objects and validate | Peter Wu | 1 | -4/+25 | |
2014-04-30 | StreamImp: support gzip compression | Peter Wu | 1 | -1/+17 | |
May yield huge bandwidth savings and improve throughput for slow links. | |||||
2014-04-30 | [WIP] Stream processor | Peter Wu | 2 | -0/+229 | |
2014-04-30 | Fix names | Peter Wu | 1 | -5/+4 | |
2014-04-30 | Merge remote-tracking branch 'origin/master' | Peter Wu | 1 | -14/+21 | |
2014-04-30 | Don't use v1 API for OAuth check | Peter Wu | 1 | -1/+1 | |
2014-04-30 | Renamed OutputStream.java because it was part of the Java classes. | Maurice Laveaux | 1 | -14/+21 | |