vd(1)                                           Quick Reference Guide                                           vd(1)

NAME
     VisiData — a terminal utility for exploring and arranging tabular data

SYNOPSIS
     vd [options] [input ...]
     vd [options] --play cmdlog [-w waitsecs] [--batch] [-o output] [field=value ...]

DESCRIPTION
     VisiData is a multipurpose tool for exploring, cleaning, editing, and restructuring data.  Rows can be selected,
     filtered, and grouped; columns can be rearranged, transformed, and derived via regex or Python expressions;
     workflows can be saved, documented, and replayed.  VisiData is built on the vdtui platform.

   REPLAY MODE
     -p, --play=cmdlog       replays a saved cmdlog within the interface
     -w, --replay-wait=seconds
                             waits seconds between commands
     -b, --batch             replays in batch mode (with no interface)
     -o, --output=file       saves final visible sheet to file as .tsv
     --replay-movement=bool  toggles --play to move cursor cell-by-cell
     field=value             replaces "{field}" in cmdlog contents with value

   Commands During Replay
        ^U                   pauses/resumes replay
        Space                executes next row in replaying sheet
        ^K                   cancels current replay

   GLOBAL COMMANDS
     In most cases, commands that affect 'selected rows' will affect all rows if no rows are selected.

   Keystrokes For The Cautious
      F1   z?         views this man page
     gF1  gz?         views all commands available on current sheet
     ^Q               aborts program immediately
     ^C               cancels user input or current task
      q               quits current sheet
     gq               quits all sheets (clean exit)

   Cursor Movement
     Arrow PgUp Home  moves as expected
      h   j   k   l   moves left/down/up/right
     gh  gj  gk  gl   moves all the way to the left/bottom/top/right
          G  gg       moves all the way to the bottom/top
     ^B  ^F           scrolls one page back/forward

      /   ? regex     searches for regex forward/backward in current column
     g/  g? regex     searches for regex forward/backward over all visible columns
      n   N           moves to next/previous match from last search

      <   >           moves up/down the current column to the next value
      {   }           moves up/down the current column to the next selected row

      c regex         moves to the next column with name matching regex
      r regex         moves to the next row with key matching regex
     zc  zr number    moves to given column/row number

      H   J   K   L   slides current row/column left/down/up/right
     gH  gJ  gK  gL   slides current row/column all the way to the left/bottom/top/right of sheet

     zh  zj  zk  zl   scrolls one left/down/up/right
       zt  zz  zb     scrolls current row to top/center/bottom of screen

   Column Manipulation
      _ (underscore)  adjusts width of current column
     g_               adjusts width of all visible columns
     z_ number        adjusts width of current column to number

      - (hyphen)      hides current column (to unhide, go to Columns sheet and edit its width)
     z-               cuts width of current column in half

     !                pins current column on the left as a key column
     ^                edits name of current column
     ~  #  %  $  @    sets type of current column to str/int/float/currency/date

       = expr         creates new column from Python expr, with column names as variables
      g= expr         sets current column for selected rows to result of Python expr
     gz= iterable     sets selected rows in current column to the results of a Python iterable
      z= expr         evaluates Python expr on current row and displays result on status line

      g^              sets names of all visible columns to contents of current row
      z^              sets name of current column to contents of current cell
     gz^              sets name of current column to combined contents of current cell in selected rows

      ' (tick)        adds a frozen copy of current column with all cells evaluated
     g'               opens a frozen copy of current sheet with all visible columns evaluated

     : regex          adds new columns from regex split; number of columns determined by example row at cursor
     ; regex          adds new columns from capture groups of regex; also requires example row
     * regex/subst    replaces regex with subst, which may include backreferences (\1 etc)

   Row Selection
      s   t   u       selects/toggles/unselects current row
     gs  gt  gu       selects/toggles/unselects all rows
      |   \ regex     selects/unselects rows matching regex in current column
     g|  g\ regex     selects/unselects rows matching regex in any visible column
      , (comma)       selects rows matching current cell in current column
     g,               selects rows matching this row in all visible columns

   Row Sorting/Filtering
      [   ]           sorts ascending/descending by current column
     g[  g]           sorts ascending/descending by all key columns
      "               opens duplicate sheet with only selected rows
     g"               opens duplicate sheet with all rows
     gz"              opens duplicate sheet with deepcopy of selected rows

   Editing Rows and Cells
       a              appends a blank row
      ga number       appends number blank rows
       d  gd          deletes current/all selected row(s) and moves to clipboard
       y  gy          copies current/all selected row(s) to clipboard
       p   Shift-P    pastes most recent clipboard rows after/before current row
       f              fills null cells in current column with content of non-null cells up the current column
       e text         edits contents of current cell
      ge text         sets contents of current column for selected rows to text
      zd   Del        sets contents of current cell to None
     gzd  gDel        sets contents of cells in current column to None for selected rows

     Commands While Editing Input
        Enter  ^C        accepts/aborts input
        ^O               opens external $EDITOR to edit contents
        ^R               reloads initial value
        ^A  ^E           moves to beginning/end of line
        ^B  ^F           moves back/forward one character
        ^H  ^D           deletes previous/current character
        ^T               transposes previous and current characters
        ^U  ^K           clears from cursor to beginning/end of line
        Backspace  Del   deletes previous/current character
        Insert           toggles insert mode
        Up  Down         sets contents to previous/next in history
        Tab  Shift-Tab   autocompletes input (when available)

   Data Toolkit
      o input         opens input in VisiData
     ^S filename      saves current sheet to filename in format determined by extension (default .tsv)
     ^D filename.vd   saves commandlog to filename.vd file
     Shift-A number   opens new blank sheet with number columns
     Shift-R number   opens duplicate sheet with a random population subset of number rows

      + aggregator    adds aggregator to current column (see Frequency Table)
     z+ aggregator    displays result of aggregator over values in selected rows for current column

   Data Visualization
      . (dot)       graphs current numeric column vs key columns. Numeric key column is on the x-axis, while
                    categorical key columns determine color.
     g.             opens a graph of all visible numeric columns vs key column

     If rows on the current sheet represent plottable coordinates (as in .shp or vector .mbtiles sources), . plots
     the current row, and g. plots all selected rows (or all rows if none selected).

     Canvas-specific Commands
         +   -              increase/decrease zoomlevel, centered on cursor
         s   u              selects/unselects rows on source sheet contained within canvas cursor
        gs  gu              selects/unselects rows visible on the screen
        Enter               pushes source sheet with only rows contained within canvas cursor
        1 - 9               toggles display of layers
        mouse scrollwheel   zooms in/out on a canvas
        left click-drag     sets canvas cursor
        right click-drag    scrolls canvas

   Other Commands
     Shift-V          views contents of current cell in a new TextSheet
        w                toggles text wrap (only on TextSheet)
      ^^ (Ctrl-^)     jumps to previous sheet (swaps with current sheet)
      ^E              views traceback for most recent error
     g^E              views traceback for most recent errors
     z^E              views traceback for error in current cell
      ^L              refreshes screen
      ^G              shows cursor position and bounds of current sheet on status line
      ^V              shows version information on status line
      ^R              reloads current sheet
     z^R              clears cache for current column
      ^Z              sends a SIGSTOP

     ^P               opens Status History
     ^X               evalutes Python expression and opens sheet for browsing resulting Python object
     ^Y  z^Y          opens sheet of current row/cell as Python object

   Internal Sheets List

     Metasheets
      .  Columns Sheet (Shift-C)     edit column properties
      .  Sheets Sheet (Shift-S)      jump between sheets or join them together
      .  Options Sheet (Shift-O)     edit all configurable options
      .  Commandlog (Shift-D)        modify and save commands for replay
      .  Error Sheet (^E)            view last error
      .  Status History (^P)         view history of status messages
      .  Tasks Sheet (^T)            view, cancel, and profile asynchronous tasks

     Derived Sheets
      .  Frequency Table (Shift-F)   group rows by column value, with aggregations of other columns
      .  Describe Sheet (Shift-I)    view summary statistics for each column
      .  Pivot Table (Shift-W)       group rows by key and summarize current column
      .  Melted Sheet (Shift-M)      unpivot non-key columns into variable/value columns

   METASHEETS
   Columns Sheet (Shift-C)
     Properties of columns on the source sheet can be changed with standard editing commands (e ge g= Del) on the
     Columns Sheet. Multiple aggregators can be set by listing them (separated by spaces) in the aggregators column.
     The 'g' commands affect the selected rows, which are actually the literal columns on the source sheet.
     (sheet-specific commands)
         &               adds column from concatenating selected source columns
         !               toggles current column as key on source sheet
        g!               toggles selected columns as keys on source sheet
        g+               adds aggregator to selected source columns
        g_ (underscore)  adjusts widths of selected columns on source sheet
        g- (hyphen)      hides selected columns on source sheet
         ~  #  %  $  @   sets type of current column to str/int/float/currency/date
        g~ g# g% g$ g@   sets type of selected columns to str/int/float/currency/date
         Enter           opens a Frequency Table sheet grouped on column referenced in current row

   Sheets Sheet (Shift-S)
     (sheet-specific commands)
        Enter            jumps to sheet referenced in current row
        & jointype       merges selected sheets with visible columns from all, keeping rows according to jointype:
                         .  inner  keeps only rows which match keys on all sheets
                         .  left   keeps all rows from first selected sheet
                         .  full   keeps all rows from all sheets (union)
                         .  diff   keeps only rows NOT in all sheets
                         .  append keeps all rows from all sheets (concatenation)

   Options Sheet (Shift-O)
     (sheet-specific commands)
        Enter            edits option

   Commandlog (Shift-D)
     (sheet-specific commands)
         ^Z              sends a SIGSTOP
         x               replays command in current row
        gx               replays contents of entire commandlog

   DERIVED SHEETS
   Frequency Table (Shift-F)
     A Frequency Table groups rows by one or more columns, and includes summary columns for those with aggregators.
     (global commands)
        gF               opens a Frequency Table, grouped by all key columns on source sheet
        zF               opens a one-line summary for selected rows
        (sheet-specific commands)
         s   t   u       selects/toggles/unselects these entries in source sheet

   Describe Sheet (Shift-I)
     (sheet-specific commands)
        zs  zt  zu       selects/toggles/unselects rows on source sheet which are being described in current cell
         Enter           opens a Frequency Table sheet grouped on column referenced in current row
        zEnter           opens copy of source sheet with rows described in current cell

   Pivot Table (Shift-W)
     Set key column(s) and aggregators on column(s) before pressing Shift-W on the column to pivot.
     (sheet-specific commands)
         Enter           opens sheet of source rows which comprise current pivot row
        zEnter           opens sheet of source rows which comprise current pivot cell

   Melted Sheet (Shift-M)
     Opens melted sheet (unpivot), with all non-key columns reduced to Variable-Value rows.

   Python Object Sheet (^X ^Y)
     (sheet-specific commands)
        Enter            dives further into Python object
        e                edits contents of current cell
        w                toggles whether methods and hidden properties are shown

OPTIONS
     -f, --filetype=filetype      tsv                uses loader for filetype instead of file extension
     -y, --confirm-overwrite=F    True               overwrites existing files without confirmation
     --diff=base                  None               adds colorizer for all sheets against base
     --encoding=str               utf-8              as passed to codecs.open
     --encoding-errors=str        surrogateescape    as passed to codecs.open
     --regex-flags=str            I                  flags to pass to re.compile() [AILMSUX]
     --default-width=int          20                 default column width
     --wrap=bool                  False              wrap text to fit window width on TextSheet
     --cmd-after-edit=str         j                  command keystroke to execute after successful edit
     --cmdlog-longname=bool       False              Use command longname in cmdlog if available
     --none-is-null=bool          True               if Python None counts as null
     --empty-is-null=bool         False              if empty string counts as null
     --false-is-null=bool         False              if Python False counts as null
     --zero-is-null=bool          False              if integer 0 counts as null
     --force-valid-colnames=bool  False              clean column names to be valid Python identifiers
     --debug=bool                 False              exit on error and display stacktrace
     --curses-timeout=int         100                curses timeout in ms
     --force-256-colors=bool      False              use 256 colors even if curses reports fewer
     --note-pending=str           ⌛                  note to display for pending cells
     --note-format-exc=str        ?                  cell note for an exception during type conversion or formatting
     --note-getter-exc=str        !                  cell note for an exception during computation
     --scroll-incr=int            3                  amount to scroll with scrollwheel
     --skip=int                   0                  skip first N lines of text input
     --profile-tasks=bool         True               profile async tasks
     --min-memory-mb=int          0                  minimum memory to continue loading and async processing
     --confirm-overwrite=bool     True               whether to prompt for overwrite confirmation on save
     --header=int                 1                  parse first N rows of .csv/.tsv as column names
     --delimiter=str                                 delimiter to use for tsv filetype
     --filetype=str                                  specify file type
     --pyobj-show-hidden=bool     False              show methods and _private properties
     --replay-wait=float          0.0                time to wait between replayed commands, in seconds
     --replay-movement=bool       False              insert movements during replay
     --regex-maxsplit=int         0                  maxsplit to pass to regex.split
     --show-graph-labels=bool     True               show axes and legend on graph
     --plot-colors=str                               list of distinct colors to use for plotting distinct objects
     --zoom-incr=float            2.0                amount to multiply current zoomlevel by when zooming
     --csv-dialect=str            excel              dialect passed to csv.reader
     --csv-delimiter=str          ,                  delimiter passed to csv.reader
     --csv-quotechar=str          "                  quotechar passed to csv.reader
     --csv-skipinitialspace=bool  True               skipinitialspace passed to csv.reader
     --fixed-rows=int             1000               number of rows to check for fixed width columns

   DISPLAY OPTIONS
     Display options can only be set via the Options Sheet or a config file (see FILES).

     disp_none                              visible contents of a cell whose value was None
     disp_date_fmt       %Y-%m-%d           default fmtstr to strftime for date values
     disp_truncator      …                  indicator that the contents are only partially visible
     disp_oddspace       ·                  displayable character for odd whitespace
     disp_unprintable    .                  substitute character for unprintables
     disp_column_sep     |                  separator between columns
     disp_keycol_sep     ‖                  separator between keys and rest of columns
     disp_status_fmt     {sheet.name}|      status line prefix
     disp_status_sep      |                 separator between statuses
     disp_edit_fill      _                  edit field fill character
     disp_more_left      <                  header note indicating more columns to the left
     disp_more_right     >                  header note indicating more columns to the right
     disp_error_val      ¿                  displayed contents for computation exception
     disp_ambig_width    1                  width to use for unicode chars marked ambiguous
     color_default       normal             the default color
     color_default_hdr   bold underline     color of the column headers
     color_current_row   reverse            color of the cursor row
     color_current_col   bold               color of the cursor column
     color_current_hdr   reverse underline  color of the header for the cursor column
     color_column_sep    246 blue           color of column separators
     color_key_col       81 cyan            color of key columns
     color_selected_row  215 yellow         color of selected rows
     color_status        bold               status line color
     color_edit_cell     normal             edit cell color
     disp_pending                           string to display in pending cells
     color_note_pending  bold magenta       color of note of pending cells
     color_note_type     226 green          cell note for numeric types in anytype columns
     color_format_exc    48 bold yellow     color of formatting exception note
     color_getter_exc    red bold           color of computation exception note
     disp_histogram      *                  histogram element character
     disp_histolen       80                 width of histogram column
     disp_replay_play    ▶                  status indicator for active replay
     disp_replay_pause   ‖                  status indicator for paused replay
     disp_pixel_random   False              randomly choose attr from set of pixels instead of most common
     color_graph_axis    bold               color for graph axis labels

EXAMPLES
           vd foo.tsv

           vd -f sqlite bar.db

           vd postgres://username:password@hostname:port/database

           vd --play tests/pivot.vd --replay-wait 1 --output tests/pivot.tsv

           ls -l | vd -f fixed --skip 1 --header 0

           vd --diff foo.tsv bar.tsv

FILES
     At the start of every session, VisiData looks for, in order, $PWD/.visidatarc, $XDG_CONFIG_HOME/visidata/config,
     or $HOME/.visidatarc, and calls Python exec() on its contents. For example:

        options.min_memory_mb=100  # stop processing without 100MB free

        globalCommand('0', 'gh')   # alias '0' to move to first column, like vim

        def median(values):
            L = sorted(values)
            return L[len(L)//2]

        aggregator('median', median)

SUPPORTED SOURCES
     These are the supported sources:

        tsv (tab-separated value)
           Plain and simple. VisiData writes tsv format by default. See the --delimiter option.

        csv (comma-separated value)
           .csv files are a scourge upon the earth, and still regrettably common. See the --csv-dialect,
           --csv-delimiter, --csv-quotechar, and --csv-skipinitialspace options.

        fixed (fixed width text)
           Columns are autodetected from the first 1000 rows (adjustable with --fixed-rows).

     For these multi-table sources, the first sheet is a directory of tables.
     Enter loads the entire table into memory.

        sqlite
        xlsx (requires openpyxl)
        xls (requires xlrd)
        hdf5 (requires h5py)
        postgres (requires psycopg2)
        shp (requires pyshp)
        mbtiles (vector only, requires mapbox-vector-tile)

AUTHOR
     VisiData was conceived and implemented by Saul Pwanson <vd@saul.pw>.

Linux/MacOS                                        October 5, 2017                                        Linux/MacOS