rawdog: RSS Aggregator Without Delusions Of Grandeur
Adam Sampson <ats@offog.org>
-rawdog is an RSS (and other) feed aggregator, based on Mark Pilgrim's flexible
-feed parser. It's just an aggregator; it's not a weblog authoring tool, nor is
-it an NNTP gateway, outliner, mailserver or anything else. rawdog probably
-only runs on Unix-like systems.
+rawdog is an extensible feed aggregator for Unix-like systems, based on
+the Python feedparser module (originally developed by Mark Pilgrim).
(Important: If you're upgrading from rawdog 1.x to rawdog 2.x, please
read the NEWS file to find out how to convert your rawdog state file.)
installation, so if you're on a Debian system you'll need to install the
"python-dev" package first.
-rawdog reads articles from a number of feeds and writes out a single
-HTML file, based on a template either provided by the user or generated
-by rawdog, containing the latest articles it's seen. It uses the ETags
-and Last-Modified headers to avoid fetching a file that hasn't changed,
-and supports gzip compression to reduce bandwidth when it has. It is
-configured from a simple text file; the only state kept between
-invocations that can't be reconstructed from the feeds is the ordering
-of articles.
-
-To install rawdog on your system, use distutils -- "python setup.py install".
-This will install the library modules that rawdog needs, and will install the
-"rawdog" binary that you can use to run it. (If you want to install to a
-non-standard prefix, read the help provided by "python setup.py install
---help".)
-
-rawdog needs a config file to function. Make the directory ".rawdog" in your
-$HOME directory, copy the provided file "config" into that directory, and edit
-it to suit your preferences. (Comments in that file describe what each of the
-options does.) You should copy the provided file "style.css" into the same
-directory that you've told rawdog to write its HTML output to. (rawdog should
-be usable from a browser that doesn't support CSS, but it won't be very
-pretty.)
-
-When you invoke rawdog from the command line, you give it a series of actions
-to perform -- for instance, "rawdog --update --write" tells it to do the
-"--update" action, then the "--write" action. The actions supported are
-as follows:
-
-"--update" (or "-u"): Fetch data from the feeds and store it. This could
-take some time if you've got lots of feeds.
-
-"--write" (or "-w"): Write out the HTML output file.
-
-"--list" (or "-l"): List brief information about each of the feeds that
-was known about when "--update" was last done.
-
-"--update-feed SOMEURL" (or "-f SOMEURL"), where SOMEURL is the URL of a
-known feed: Update that feed immediately, even if its period hasn't
-elapsed since it was last updated. This is useful if you're trying to
-debug your own feed.
-
-"--config FILE" (or "-c FILE"), where FILE is an absolute path or a path
-relative to your .rawdog directory: Read FILE as an additional config
-file; any options provided in FILE will override those already set in
-the main config file (with the exception of "feed", which is
-cumulative). Note that $HOME/.rawdog/config will still be read first
-even if you specify this option. This is useful if you want rawdog to
-write two different output files with different sets of options ("rawdog
--u -w -c config2 -w" will first update and write with the main config
-file, then read config2, then write again).
-
-"--show-template" (or "-t"): Print the template currently in use to
-stdout. This is useful as a starting point if you want to modify your
-own template: do "rawdog -t >~/.rawdog/mytemplate" with "template
-default" in your config file, and you'll get a copy of the default
-template to edit.
-
-"--show-itemtemplate" (or "-T"): As for "--show-template", but for the
-item template.
-
-"--add URL" (or "-a URL"): Add a new feed to the config file. This uses
-Mark Pilgrim's "feedfinder" module to try to figure out a feed for any
-given URL, so you can usually just give it the URL of a blog's main
-page, and it'll automatically detect the appropriate feed. (It'll tell
-you if it can't guess a feed for the URL you give it.)
-
-There are also the following options which may only be supplied once
-(they're read before any of the actions are performed):
-
-"--help": Provide a brief summary of all the options rawdog supports,
-and exit.
-
-"--dir DIR" (or "-d DIR"), where DIR is a directory: Use DIR instead of
-the $HOME/.rawdog directory. This is useful if you want to have two or
-more completely different rawdog setups with different sets of feeds;
-just create a directory for each.
-
-"--verbose" (or "-v"): Print more information about what rawdog's doing
-while it's working. This is useful for tracking down problems.
-
-"--no-locking" (or "-N"): Don't attempt to lock the state file. rawdog
-usually claims a lock on the state file so that you can't have more than
-one instance of rawdog running at the same time. Unfortunately, some
-operating systems and filesystems don't support file locking; you can
-use this option to disable locking if you're in that situation.
-
-"--no-lock-wait" (or "-W"): If the state file lock can't be claimed
-immediately, exit silently. Normally, rawdog will wait until it can
-claim the lock, then run as usual; however, if you've got a very large
-number of feeds and a slow machine or network connection, and you run
-rawdog periodically from cron, then you may find you often have a number
-of rawdog processes competing for the lock. In that situation, use this
-option so that you've really only ever got one rawdog process running at
-once.
-
-"--upgrade OLDDIR NEWDIR": This option can be used to convert your state
-file when upgrading from rawdog 1.x to rawdog 2.x; see the NEWS file for
-more information. If you're not doing this, then it won't be of any use
-to you.
+To install rawdog on your system, use distutils -- "python setup.py
+install". This will install the library modules that rawdog needs, and
+will install the "rawdog" command that you can use to run it. (If you
+want to install to a non-standard prefix, read the help provided by
+"python setup.py install --help".)
+
+rawdog needs a config file to function. Make the directory ".rawdog" in
+your $HOME directory, copy the provided file "config" into that
+directory, and edit it to suit your preferences. Comments in that file
+describe what each of the options does.
+
+You should copy the provided file "style.css" into the same directory
+that you've told rawdog to write its HTML output to. rawdog should be
+usable from a browser that doesn't support CSS, but it won't be very
+pretty.
+
+When you invoke rawdog from the command line, you give it a series of
+actions to perform -- for instance, "rawdog --update --write" tells it
+to do the "--update" action (downloading articles from feeds), then the
+"--write" action (writing the latest articles it knows about to the HTML
+file).
+
+For details of all rawdog's actions and command-line options, see the
+rawdog(1) man page -- "man rawdog" after installation.
You will want to run "rawdog -uw" periodically to fetch data and write
the output file. The easiest way to do this is to add a crontab entry
start.) This will run rawdog every ten minutes.
If you want rawdog to fetch URLs through a proxy server, then set your
-"http_proxy" environment variable appropriately; depending on your version of
-cron, putting something like:
+"http_proxy" environment variable appropriately; depending on your
+version of cron, putting something like:
http_proxy=http://myproxy.mycompany.com:3128/
-at the top of your crontab should be appropriate. (The http_proxy variable will
-work for many other programs too.)
+at the top of your crontab should be appropriate. (The http_proxy
+variable will work for many other programs too.)
-In the event that rawdog gets horribly confused (for instance, if your system
-clock has a huge jump and it thinks it won't need to fetch anything for the
-next thirty years), you can forcibly clear its state by removing the
-~/.rawdog/state file (and the ~/.rawdog/feeds/*.state files, if you've
-got the "splitstate" option turned on).
+In the event that rawdog gets horribly confused (for instance, if your
+system clock has a huge jump and it thinks it won't need to fetch
+anything for the next thirty years), you can forcibly clear its state by
+removing the ~/.rawdog/state file (and the ~/.rawdog/feeds/*.state
+files, if you've got the "splitstate" option turned on).
-If you don't like the appearance of rawdog, then customise the style.css file.
-If you come up with one that looks much better than the existing one, please
-send it to me!
+If you don't like the appearance of rawdog, then customise the style.css
+file. If you come up with one that looks much better than the existing
+one, please send it to me!
This should, hopefully, be all you need to know. If rawdog breaks in
-interesting ways, please tell me at the email address at the top of this file.
+interesting ways, please tell me at the email address at the top of this
+file.
.B rawdog
.RI [ options ]
.SH DESCRIPTION
-This manual page briefly documents the
-.B rawdog
-command.
+\fBrawdog\fP is a feed aggregator for Unix-like systems.
+.PP
+\fBrawdog\fP uses the Python \fBfeedparser\fP module to retrieve
+articles from a number of feeds in RSS, Atom and other formats, and
+writes out a single HTML file, based on a template either provided by
+the user or generated by \fBrawdog\fP, containing the latest articles
+it's seen.
.PP
-\fBrawdog\fP is an RSS (and other) feed aggregator, based on Mark
-Pilgrim's flexible feed parser. \fBrawdog\fP reads articles from a
-number of feeds and writes out a single HTML file, based on a template
-either provided by the user or generated by rawdog, containing the
-latest articles it's seen.
+\fBrawdog\fP uses the ETags and Last-Modified headers to avoid fetching
+a file that hasn't changed, and supports gzip and delta compression to
+reduce bandwidth when it has. \fBrawdog\fP is configured from a simple
+text file; the only state kept between invocations that can't be
+reconstructed from the feeds is the ordering of articles.
.SH OPTIONS
This program follows the usual GNU command line syntax, with long
options starting with two dashes (`\-'). A summary of options is
Use
.I DIR
instead of the $HOME/.rawdog directory.
+This option lets you have two or more \fBrawdog\fP setups with different
+configurations and sets of feeds.
.TP
.B \-N, \-\-no\-locking
Do not lock the state file.
+.IP ""
+\fBrawdog\fP usually claims a lock on its state file, to stop more than
+one instance from running at the same time. Unfortunately, some
+filesystems don't support file locking; you can use this option to
+disable locking entirely if you're in that situation.
.TP
.B \-v, \-\-verbose
-Print more detailed status information.
+Print more detailed information about what \fBrawdog\fP is doing while
+it runs.
.TP
.B \-W, \-\-no\-lock\-wait
-Exit silently if the state file is locked.
+Exit silently if the state file is already locked.
+.IP ""
+If the state file is already locked, \fBrawdog\fP will normally wait
+until it becomes available, then run. However, if you're got a lot of
+feeds and a slow network connection, you might prefer \fBrawdog\fP to
+just give up immediately if the previous instance is still running.
.SS Actions
\fBrawdog\fP will perform these actions in the order given.
.TP
Try to find a feed associated with
.I URL
and add it to the config file.
+.IP ""
+.I URL
+may be a feed itself, or it can be an HTML page that links to a feed in
+any of a variety of ways. \fBrawdog\fP uses heuristics to pick the best
+feed it can find, and will complain if it can't find one.
.TP
.BI \-c \ FILE\fP, \ \-\-config \ FILE
Read
"feed", which is cumulative).
.I FILE
may be an absolute path or a path relative to your .rawdog directory.
+.IP ""
Note that $HOME/.rawdog/config will still be read first even if you
-specify this option.
+specify this option. \fB\-c\fP is mostly useful when you want to write
+the same set of feeds out using two different sets of output options.
.TP
.BI \-f \ URL\fP, \ \-\-update\-feed \ URL
Update the feed pointed to by
.I URL
immediately, even if its period hasn't elapsed since it was last
-updated.
+updated. This is useful when you're publishing a feed yourself, and want
+to test whether it's working properly.
.TP
.B \-l, \ \-\-list
-List feeds known at the time of last update.
+List brief information about each of the feeds that was known about at
+the time of the last update.
.TP
.BI \-r \ URL\fP, \ \-\-remove \ URL
Remove feed
from the config file.
.TP
.B \-t, \ \-\-show\-template
-Print the
-.I template
-currently in use to stdout.
+Print the page template currently in use to stdout. This can be used as
+a starting point if you want to design your own template for use with
+the \fBtemplate\fP option in the config file.
.TP
.B \-T, \ \-\-show\-itemtemplate
-Print the
-.I itemtemplate
-currently in use to stdout.
+Print the item template currently in use to stdout.
.TP
.B \-u, \-\-update
Fetch data from the feeds and store it. This could take some time if
that action, then exit.
.TP
.B \-\-help
-Provide a brief summary of all the options rawdog supports.
+Provide a brief summary of all the options \fBrawdog\fP supports.
.TP
.BI \-\-upgrade \ OLDDIR\ NEWDIR
-Import feed state from rawdog 1.x directory
+Import feed state from \fBrawdog\fP 1.x directory
.I OLDDIR
-into rawdog 2.x directory
+into \fBrawdog\fP 2.x directory
.IR NEWDIR .
.SH EXAMPLES
\fBrawdog\fP is typically invoked from
.PP
.nf
.RS
-0 * * * * rawdog -Wvw
+0 * * * * rawdog -Wuw
.RE
.fi
.SH FILES
.SH SEE ALSO
.BR cron (1).
.SH AUTHOR
-
rawdog was mostly written by Adam Sampson <ats@offog.org>, with
-contributions and bug reports from many of rawdog's users.
-See rawdog's NEWS file for a complete list of contributors.
+
\fBrawdog\fP was mostly written by Adam Sampson <ats@offog.org>, with
+contributions and bug reports from many of \fBrawdog\fP's users.
+See \fBrawdog\fP's NEWS file for a complete list of contributors.
.PP
This manual page was originally written by Decklin Foster
<decklin@red\-bean.com>, for the Debian project (but may be used by others).
projects / rawdog / .git / commitdiff