NEWSRAFT(1)

NAME

newsraft - feed reader for terminal

SYNOPSIS

newsraft [-f FILE1] [-c FILE2] [-d FILE3] [-l FILE4] [-e ACTION] [-v] [-h]

DESCRIPTION

Newsraft is a small text based program for reading syndication feeds. It obtains content from a given set of sources and lets you browse it all via one streamlined user interface.

OPTIONS

-f FILE
    Force feeds file to FILE.

-c FILE
    Force config file to FILE.

-d FILE
    Force database file to FILE.

-l FILE
    Write logs to FILE.

-e ACTION
    Execute one-time ACTION from the following list:
        convert-opml-to-feeds (takes OPML from standard input)
        convert-feeds-to-opml (takes feeds from feeds file)
        reload-all
        print-unread-items-count
        purge-abandoned

-v
    Print version information.

-h
    Print usage information.

STARTER GUIDE

To start using Newsraft you have to create a feeds file with the list of links to feeds you want to receive news from. Check out FEEDS FILE section for file syntax and valid paths.

When feeds file is ready, you can launch Newsraft. There are only 4 menus you will have to deal with: sections, feeds, items and pager. Default binds are listed in ACTIONS section.

Sections menu consists of section entries which are needed to organize feeds in groups to be able to process them in bulk. They are kind of directories for feeds. If you didn't specify any section declarations in your feeds file then you will get to the feeds menu straightaway.

Feeds menu consists of feed entries. Every feed entry contains news downloaded from one specific source which you have set in feeds file. To update a single feed you have to select it and press r or R if you want to update all feeds. From feeds menu you can get to the items menu by entering some feed.

Items menu consists of feed item entries (i. e. single pieces of news) which you get when you update feeds in the previous menu. Every feed item entry has two switchable properties - read state and importance state. Keys to change read state: d to mark read, D to mark unread, ^D to mark everything read. Keys to change importance state: f to flag important, F to flag unimportant. To view item's content you have to go to pager menu by entering selected item.

Pager menu will display some details about selected item and render its content if it was provided by feed. Usually feed item entries have a links section with one link pointing to a related web page and several links that were mentioned in the item's content. You can copy these links into your clipboard with y key and open them in your web browser with o key. To target a key action to link with a specific index you have to prefix your key with this index. For example, 5y will copy fifth link and 17o will open seventeenth link in the web browser. You can also setup custom command bindings to execute any commands with these links. Consider this config file:

    bind m exec mpv "%l"
    bind f exec feh "%l"

With this you will be able to open any link in mpv(1) and feh(1) directly from your terminal! Isn't it awesome? It is freaking amazing!

For both sections menu and feeds menu there is a special explore mode. You can toggle it by pressing the tab key. It's truly miraculous: it reveals all the news in the current context (combines news from all feeds of the current menu into one list). This mode may come in handy when you want to quickly scroll through all the news without switching between sections and feeds back and forth.

And for a dessert, I'll tell you about the search functionality. You can type / to begin search input - enter the desired query here and press Enter. This will open up an items menu with all the items matching your query in the current context.

CONFIGURATION

  FEEDS FILE

This file contains feed entries that Newsraft will display and process. There are 4 types of lines in feeds file.

Comment lines start with # character. These lines are completely ignored. For example:

    # Look closely. The beautiful may be small.

Feed lines start with a URL. After at least one whitespace character, the name of the feed may be specified - it must be enclosed in double quotes. For example:

    https://example.org/feed.xml "Lorem Ipsum Blog"

Generator lines start with a command enclosed in $(). These act just like feed lines but instead of fetching resources from a remote server they use the output of the specified command to obtain the content.

    $(cat ~/local-feed.xml) "Lorem Ipsum Blog"

Section lines start with @ character. After any number of whitespace characters, the name of the section must be specified. For example:

    @ Software Releases

Both feed and section lines allow you to set individual settings and binds for them. The syntax is as follows:

    @ Lorem Ipsum < reload-period 1440
    http://example.org/feed1.xml "Dolor Sit" < reload-period 60; item-limit 500
    http://example.org/feed2.xml "Id Est" < bind b mark-read; exec book.sh "%l"

Settings set for feeds take precedence over the settings specified for sections. Not every setting supports individual assignment - only settings with asterisk (*) on them do (see SETTINGS section).

Search precedence:
1.  $XDG_CONFIG_HOME/newsraft/feeds
2.  $HOME/.config/newsraft/feeds
3.  $HOME/.newsraft/feeds

  CONFIG FILE

This file is used to override default settings and bindings of Newsraft. Presence of config file is totally optional and Newsraft will work without it just fine. There are 3 types of lines in config file.

Comment lines start with # character. These lines are completely ignored. For example:

    # Good design is as little design as possible.

Setting lines start with a setting name and end with a setting value. Available settings are listed in the SETTINGS and COLOR SETTINGS sections. Here are a couple of examples:

    scrolloff 5000
    list-entry-date-format "%D"
    feeds-menu-paramount-explore true

Binding lines start with the bind word. They define actions that are performed when certain keys are pressed. Complete list of available actions can be found in the ACTIONS section. Here's an example:

    bind r reload-all

The special exec action makes it possible to run shell commands when the bound key is pressed. Specifiers within the command are replaced with values corresponding to the currently selected entry as per menu-item-entry-format:

    bind m exec setsid mpv --terminal=no "%l" &

The edit action runs a specified SQL query on the database, so please be careful! @selected specifier is replaced with a condition which identifies the currently selected entry - make sure to include it if you want to target individual item/feed rather than the whole database:

    bind w edit UPDATE items SET user_data = json_set(IFNULL(user_data, '{}'), '$.toWatch', 1) WHERE @selected
    bind W edit UPDATE items SET user_data = json_set(IFNULL(user_data, '{}'), '$.toWatch', 0) WHERE @selected

Use the find action to retrieve items based on a specified SQL condition in the current context. For instance, to search for items marked as toWatch (as shown in the previous example), one can use the bindings like the following:

    bind f find user_data LIKE '%"toWatch":1%'
    bind f find json_extract(user_data, '$.toWatch') = 1

Binding lines can fit multiple actions in a single key. These actions will be executed in the order in which they are specified. Assigned actions must be separated with semicolon (;) characters, for example:

    bind key action; exec command; action; edit query; action

In case you want to disable some binding which was set in Newsraft by default, you can use a line according to this format:

    unbind key

Search precedence:
1.  $XDG_CONFIG_HOME/newsraft/config
2.  $HOME/.config/newsraft/config
3.  $HOME/.newsraft/config

  DATABASE FILE

This file stores everything you download from feeds in sqlite3(1) format. Although you now know the format in which the data is stored, it is highly recommended to avoid modifying the database manually - things will break and it will be very sad.

Search precedence:
1.  $XDG_DATA_HOME/newsraft/newsraft.sqlite3
2.  $HOME/.local/share/newsraft/newsraft.sqlite3
3.  $HOME/.newsraft/newsraft.sqlite3

SETTINGS

Settings with asterisk (*) on them can be set for individual feeds and sections.

  reload-period (*)
Default: 0. Feed auto reload period in minutes. If set to 0, no auto reloads will be run.

  suppress-errors (*)
Default: false. If true, feed update error indication in the menu will be disabled. It's recommended to set this setting only for specific feeds that are expected to fail frequently and you are tired of seeing their errors.

  item-rule (*)
Default: "". Item search condition when accessing database. This can be very useful in managing feeds with a heavy spam flow: you set a condition based on some parameters and only those entries that meet this condition will be shown in the feed. It's specified in SQL format. It probably only makes sense to set this setting for individual feeds, and not globally (see FEEDS FILE section to understand how).

Available parameters:
guid | (string) | Globally unique identifier of the article which is expected to be unique within the originating feed
title | (string) | Title of the article
link | (string) | URL of the article
content | (string) | Content of the article which is stored exactly as it appears in the feed, with all original HTML preserved
attachments | (string) | Serialized string of all attachments with URL links, MIME types and byte sizes
persons | (string) | Serialized string of all people related to the article with names and email addresses
download_date | (integer) | Timestamp of download date in seconds since 1970
publication_date | (integer) | Timestamp of publication date in seconds since 1970
update_date | (integer) | Timestamp of update date in seconds since 1970
user_data | (string) | User-defined data for the item. It can be modified using the edit action (see CONFIG FILE section)

Here are some examples of correct setting values:

    title NOT LIKE '%Rust%'
    persons LIKE '%PHARMACIST%' OR persons LIKE '%OFFL1NX%'
    attachments LIKE '%audio/mp3%' OR attachments LIKE '%video/mp4%'
    CAST(strftime('%Y', publication_date, 'unixepoch') AS INTEGER) >= 2024
    NOT (title REGEXP '.+:.+\".+\".+by' OR title LIKE '%(ultra slowed)%' OR title LIKE '%(1 hour version)%')

REGEXP operator uses POSIX Extended Regular Expression syntax, see regex(7).

  item-limit (*)
Default: 0. Maximum number of items stored in a feed. If set to 0, no limit will be set.

  item-limit-unread (*)
Default: true. If true, item-limit setting will also cap unread items.

  item-limit-important (*)
Default: false. If true, item-limit setting will also cap important items.

  scrolloff
Default: 0. Minimal number of list menu entries to keep above and below the selected entry. If you set it to a very large value the selected entry will always be in the middle of the list menu (except for start and end of the list menu).

  scrollwrap
Default: false. If true, moving down while on the last item in a list will wrap around to the top and vice versa.

  pager-width (*)
Default: 100. Pager width in characters. If set to 0, the pager will take up all available space.

  pager-centering (*)
Default: true. If true and pager-width is not 0, pager will center its content horizontally.

  menu-item-sorting
Default: time-desc. Sorting order for the items menu. Available values: time-desc, time-asc, time-download-desc, time-download-asc, time-publication-desc, time-publication-asc, time-update-desc, time-update-asc, rowid-desc, rowid-asc, unread-desc, unread-asc, important-desc, important-asc, alphabet-desc, alphabet-asc.

  menu-feed-sorting
Default: none. Sorting order for the feeds menu. Available values: unread-desc, unread-asc, alphabet-desc, alphabet-asc.

  menu-section-sorting
Default: none. Sorting order for the sections menu. Available values: unread-desc, unread-asc, alphabet-desc, alphabet-asc.

  menu-responsiveness
Default: true. If true, update menu contents as soon as possible. If false, the menu will be updated only when you re-open it.

  open-in-browser-command (*)
Default: auto. Shell command for open-in-browser action. If set to auto, most operating systems will get ${BROWSER:-xdg-open} "%l" while macOS users have it set to open "%l".

  copy-to-clipboard-command
Default: auto. Shell command for copying text to clipboard. All copied data is piped into the standard input of the specified command. If set to auto, Newsraft will determine the appropriate command based on the user environment:

•   "wl-copy" if environment variable WAYLAND_DISPLAY is set
•   "xclip -selection clipboard" if environment variable DISPLAY is set
•   "pbcopy" if Newsraft was built for macOS operating system
•   "newsraft-osc-52" otherwise

"newsraft-osc-52" is a special value that doesn't run an external command but instead triggers the OSC 52 escape sequence, instructing the terminal to copy data directly to the system clipboard. This behavior is especially useful when running Newsraft over ssh(1), as it allows clipboard operations to affect the local system rather than the remote one.

  notification-command (*)
Default: auto. Shell command for invoking system notifications about new news received. If set to auto, Newsraft will determine the appropriate command based on the user environment:

•   "notify-send 'Newsraft brought %q news!'" if WAYLAND_DISPLAY or DISPLAY is set in environment
•   "osascript -e 'display notification "Newsraft brought %q news!"'" if Newsraft was built for macOS operating system
•   "printf '\e]9;Newsraft brought %q news!\a'" (OSC 9 escape sequence) otherwise

  proxy (*)
Default: "". Sets the proxy to use for the network requests. It must be either a hostname or dotted numerical IPv4 address. To specify IPv6 address you have to enclose it within square brackets. Port number can be set by appending :PORT to the end of setting value. By default proxy protocol is considered HTTP, but you can set a different one by prepending SCHEME:// to the setting value.

  proxy-user (*)
Default: "". User for authentication with the proxy server.

  proxy-password (*)
Default: "". Password for authentication with the proxy server.

  global-section-name
Default: Global. Name of the section that contains all feeds.

  global-section-hide
Default: false. If true, global section will not be shown.

  status-show-menu-path
Default: true. If true, print menu path in the status bar.

  status-placeholder
Default: r:reload R:reload-all tab:explore d:read D:unread f:important F:unimportant n:next-unread N:prev-unread p:next-important P:prev-important.

Placeholder which is put in the status bar if it's empty.

  item-content-format (*)
Default: <b>Feed</b>:&nbsp;&nbsp;%f<br>|<b>Title</b>:&nbsp;%t<br>|<b>Date</b>:&nbsp;&nbsp;%d<br>|<br>%c<br>|<br><hr>%L.

Sets the HTML format according to which the item's content will be generated. Fields are separated by | character and ONLY one specifier can be placed in each field. If an item doesn't have a value corresponding to the specifier in the field, then the entire field will not be shown. Specifiers are as follows:

    f    feed title if set, feed link otherwise;
    t    item title;
    l    item link;
    d    item date;
    a    item authors;
    c    item content;
    L    item links list.

  item-content-date-format (*)
Default: %a, %d %b %Y %H:%M:%S %z. Date format in the item's content. Specifier values correspond to the strftime(3) format.

  item-content-link-format (*)
Default: <b>[%i]</b>:&nbsp;%l<br>. Link format in the links list of item's content. %i and %l will be replaced by link index and link address respectively.

  list-entry-date-format
Default: %b %d. Date format of the list entries. Specifier values correspond to the strftime(3) format.

  menu-section-entry-format
Default: %5.0u @ %t. Format of the section list entries. Specifiers are as follows:

    i    index number;
    u    unread items count;
    t    section title.

  menu-feed-entry-format
Default: %5.0u │ %t. Format of the feed list entries. Specifiers are as follows:

    i    index number;
    u    unread items count;
    n    total items count;
    l    feed link;
    t    feed name if set, feed link otherwise.

  menu-item-entry-format
Default: " %u │ %d │ %o". Format of the item list entries. Specifiers are as follows:

    i    index number;
    u    "N" if item is unread, " " otherwise;
    d    update date formatted according to list-entry-date-format;
    D    publication date formatted according to list-entry-date-format;
    l    item link;
    t    item title;
    o    item title if set, item link otherwise;
    L    feed link;
    T    feed title;
    O    feed title if set, feed link otherwise.

  menu-explore-item-entry-format
Default: " %u │ %d │ %-28O │ %o". Format of the item list entries in explore mode. Specifiers are the same as in menu-item-entry-format.

  sections-menu-paramount-explore
Default: false. Enables explore mode in sections menu by default.

  feeds-menu-paramount-explore
Default: false. Enables explore mode in feeds menu by default.

  read-on-arrival (*)
Default: false. Mark new items as read automatically.

  mark-item-unread-on-change (*)
Default: false. Mark every item that changes on a feed update as unread.

  mark-item-read-on-hover (*)
Default: false. Mark every item that gets selected as read.

  database-batch-transactions
Default: true. Apply all changes to the database file in one big transaction after all feed updates have finished instead of using a separate transaction for each feed update. This improves update performance a lot with the downside that fetched content will not be saved if you quit Newsraft before update finishes.

  database-analyze-on-startup
Default: true. Run "ANALYZE" SQLite command on the database every time you start Newsraft. It gathers statistics about database and uses it to optimize some queries making runtime faster.

  database-clean-on-startup
Default: false. Run "VACUUM" SQLite command on the database every time you start Newsraft. It rebuilds the database file by packing it into a minimal amount of disk space. This can significantly increase startup time.

  download-timeout (*)
Default: 20. Maximum time in seconds that you allow Newsraft to download one feed. Setting to 0 disables the timeout.

  download-speed-limit (*)
Default: 0. Maximum download speed in kilobytes per second (kB/s). Setting to 0 disables the limit.

  download-max-connections
Default: 500. Maximum amount of simultaneously open connections Newsraft may hold in total. If set to 0, there is no limit. You can try to increase this value or even set it to 0 if you want to squeeze out all performance to the last drop but be aware that things can start to break at high setting values. One obvious example is getaddrinfo() starts to choke with a large number of simultaneous requests trying to resolve domain names.

  download-max-host-connections
Default: 0. Maximum amount of simultaneously open connections Newsraft may hold a single host. If set to 0, there is no limit.

  user-agent (*)
Default: auto. User-Agent header sent with download requests. If set to auto, Newsraft will generate it according to the following format:

    "newsraft/" + NEWSRAFT_VERSION + " (" + OS_NAME + ")"

OS_NAME shouldn't be a matter of privacy concern, because on most systems it contains nothing more like "Linux" or "Darwin". If you want to be sure of this, check Newsraft log to see how user-agent is set at startup.

If set to "", User-Agent header will not be sent.

  respect-ttl-element (*)
Default: true. Prevents too frequent updates for some feeds. The limit is set by the creators of the feeds in order to save traffic and resources for a very rarely updated feeds. Disabling it is strongly discouraged.

  respect-expires-header (*)
Default: true. Prevents feed updates until the expiration date of the previously downloaded information in order to save traffic and resources. Disabling it is strongly discouraged.

  send-if-none-match-header (*)
Default: true. Sends an entity tag corresponding to the previously downloaded information. If the server from which the feed is downloaded contains information with the same tag, then in order to save traffic and resources, it will reject the download request. Disabling it is strongly discouraged.

  send-if-modified-since-header (*)
Default: true. Sends a date corresponding to the last modification of previously downloaded information. If the server from which the feed is downloaded contains information with the same modification date, then in order to save traffic and resources, it will reject the download request. Disabling it is strongly discouraged.

  ignore-no-color
Default: false. If true, Newsraft will use colors regardless of whether NO_COLOR environment variable is present or not.

COLOR SETTINGS

Color settings are the same settings as above, but they take two color words (foreground and background) and optional attribute words. Available colors are default, black, red, green, yellow, blue, magenta, cyan, white and colorN (N can be a number from 0 to 255). Available attributes are bold, italic and underlined.

Color setting | Default value
color-status | green default bold
color-status-info | cyan default bold
color-status-fail | red default bold
color-list-item | default default
color-list-item-selected | (not set)
color-list-item-unread | yellow default
color-list-item-important | magenta default
color-list-feed | default default
color-list-feed-selected | (not set)
color-list-feed-unread | yellow default
color-list-feed-failed | red default
color-list-section | default default
color-list-section-selected | (not set)
color-list-section-unread | yellow default
color-list-section-failed | red default

ACTIONS

Actions | Keys
select-next | j, KEY_DOWN, ^E
select-prev | k, KEY_UP, ^Y
select-next-page | space, ^F, KEY_NPAGE
select-next-page-half | ^D
select-prev-page | ^B, KEY_PPAGE
select-prev-page-half | ^U
select-first | g, KEY_HOME
select-last | G, KEY_END
jump-to-next | J
jump-to-prev | K
jump-to-next-unread | n
jump-to-prev-unread | N
jump-to-next-important | p
jump-to-prev-important | P
next-error | e
prev-error | E
goto-feed | *
shift-west | ,
shift-east | .
shift-reset | <
sort-by-time | t
sort-by-time-download | (not set)
sort-by-time-publication | (not set)
sort-by-time-update | (not set)
sort-by-rowid | w
sort-by-unread | u
sort-by-initial | z
sort-by-alphabet | a
sort-by-important | i
enter | enter, l, KEY_ENTER, KEY_RIGHT
reload | r
reload-all | R, ^R
mark-read; jump-to-next | d
mark-unread; jump-to-next | D
mark-read-all | A
mark-unread-all | (not set)
mark-important | f
mark-unimportant | F
toggle-read | (not set)
toggle-important | (not set)
toggle-explore-mode | tab
view-errors | v
open-in-browser | o
copy-to-clipboard | y, c
start-search-input | /
clean-status | `
navigate-back | h, backspace, KEY_LEFT, KEY_BACKSPACE
quit | q
quit-hard | Q
exec man newsraft | ?

You can also bind escape key, but this can potentially lead to incomplete processing of escape sequences in the terminal on a very fast input. This is due to the special role of the escape character under the hood of terminals. Your mileage may vary.

FORMATS SUPPORT

Data formats of feeds which Newsraft recognizes. Not the whole functionality of these formats is implemented, but only the functionality that is most likely to carry the most essential information.

RSS 2.0, 1.1, 1.0, 0.94, 0.93, 0.92, 0.91, 0.9
Atom 1.0
RSS Content Module
Media RSS
DublinCore 1.1 Elements
JSON Feed

ENVIRONMENT

Newsraft's behavior depends on the environment variables set, however not all environment variables affect Newsraft directly - many environment variables affect libraries that Newsraft is built upon. For example, libcurl(3) recognizes a large number of different environment variables which you can learn more about on libcurl-env(3).

XDG_CONFIG_HOME | Directory in which user-specific configuration files are stored.
XDG_DATA_HOME | Directory in which user-specific data files are stored.
HOME | User home directory.
BROWSER | User web browser.
WAYLAND_DISPLAY | Identifier of the Wayland graphics display.
DISPLAY | Identifier of the X graphics display.
NO_COLOR | Makes the interface monochrome when present.

SEE ALSO

mpv(1), feh(1), sqlite3(1), regex(7), ssh(1), strftime(3), libcurl(3), libcurl-env(3)

BUGS

Don't be ridiculous...

AUTHORS

Grigory Kirillov and contributors