Psty is a little server you run on your desktop as a sidekick to Pigshell. While pigshell can be used standalone, psty (the p is silent, as in Psmith) adds so much to its power and reach as to be practically indispensable.
Psty implements three services:
Together, these features enable powerful data movement and transformation pipelines. For instance:
pig> cat http://pigshell.com/sample/oslogos.png | wsh convert -implode 1 - - | to -g canvas
We grabbed an image from pigshell.com using the proxy, piped it via websocket to the local Unix host running psty, ran the ImageMagick convert tool to transform the image (taking care to use stdin/stdout), piped it back to pigshell and displayed the transformed image.
Psty is implemented as a single standalone Python file weighing < 1000 lines, and requires only a standard Python 2.7 distribution. It can run on Linux and Mac OS. Psty is reported to partially work (local file and proxy services) on Windows under Cygwin.
Download psty and run it from a shell as follows:
bash$ python psty.py -a -d /some/scratch/directory
This command starts all three services and exposes /some/scratch/directory to all pigshell instances running in your browser. There are CLI options to start a subset of the services, and to change the default port of 50937.
In your pigshell tab, type the command
pig> mount http://localhost:50937/ /home
and you should be able to see the contents of /some/scratch/directory inside /home.
This mount command needs to be typed every time you start or reload the page. To do it automatically,
pig> echo "HOME=/home\nmount http://localhost:50937/ $HOME" >/local/rc.sh
/local/rc.sh is a script stored in the browser's LocalStorage and will be invoked every time http://pigshell.com is (re)loaded.
Currently, pigshell assumes that websocket and proxy services are available at localhost:50937. This will be configurable in future.
Data movement: Back up all Picasa photos to a local directory
pig> mkdir /home/taj; cp /picasa/albums/TajMahal/* /home/taj
Data retrieval: Copy a file from the web, continuing from where we left off
pig> cp -c http://ftp.freebsd.org/pub/FreeBSD/ISO-IMAGES-amd64/10.0/FreeBSD-10.0-RC4-amd64-bootonly.iso /home
Running commands on the desktop:
pig> wsh ps ax
ps on your desktop and dumps the output in pigshell.
Running pipelines on the desktop:
pig> wsh sh -c 'ls | grep foo'
runs a Bourne shell on the desktop with the given pipeline. Note that
wsh 'ls | grep foo' won't work.
Visualization: Local disk usage visualized in an interactive zoomable treemap
pig> wsh du /Users/foo | to -g text | iframe -g /templates/d3-du-treemap
(Note that `du` of a deep tree may take a while, try with a shallow directory tree first)
Pigshell wants to be a common platform where web and local data can copied, mixed, viewed, analyzed and visualized using the browser's display engine. There are several issues holding it back from this goal:
Unfortunately, CORS support in the wild is practically non-existent. The Wikipedia article on CORS does not have a CORS header. Even those sites which claim to support CORS often have half-hearted support (GET but not HEAD). Picasa APIs work cross-origin as long as you're doing GETs, but don't work with POSTs, since they don't implement OPTIONS properly.
Accessing the local filesystem via an HTTP server gives us many benefits: streaming reads and writes, the ability to move data back and forth from the cloud to the desktop, and an unlimited workspace.
While some of these requirements can be met by a cloud backend, we strongly prefer pure client-side, self-scaling architectures (basically we don't want the headache of maintaining a server) Psty lets us do all these things while still being easy to deploy and use.
Psty serves data only to localhost, and then only when it sees an origin header from http://pigshell.com.
Any modification of psty can be dangerous. Specifically,
As pigshell is open source, you can verify that the code does what it is supposed to do. Running pigshell scripts from untrusted sources is as dangerous as running untrusted shell scripts on your desktop.