2011-07-28 18:47:43 +00:00
======
oyster
======
oyster is a service for tracking regularly-accessed pages, a sort of proactive cache.
2011-09-20 21:15:15 +00:00
Oyster intends to provide a command line client for interacting with the list of tracked documents and web frontend for viewing the status and retrieving data. Behind the scenes it uses a celery queue to manage the documents it is tasked with keeping up to date.
2011-07-28 18:47:43 +00:00
oyster was created by James Turk for `Sunlight Labs <http://sunlightlabs.com> `_ .
Source is available via `GitHub <http://github.com/sunlightlabs/oyster/> `_
Installation
============
2011-09-21 16:05:23 +00:00
oyster is available on PyPI: `oyster <http://pypi.python.org/pypi/oyster> `_ .
The recommended way to install oyster is to simply `` pip install oyster ``
2011-07-28 18:47:43 +00:00
Requirements
------------
* python 2.7
2012-02-13 23:39:03 +00:00
* mongodb 2.0
* pymongo 2.0
* scrapelib 0.5+
2011-09-20 15:01:58 +00:00
Usage
=====
2011-09-20 15:43:44 +00:00
* Run celeryd with beat `` celeryd -B --config=oyster.celeryconfig ``
* Run oyster HTTP portal `` python oyster/web.py ``
2011-09-20 20:16:55 +00:00
* Use oyster.client.Client to add new documents & query the store
