1. bipython 0.1.0

    bipython
logo

    the boldly indiscriminate python interpreter

    "...because you shouldn't have to choose."

    PROLOGUE

    Two interpreters, both alike in dignity,
    In fair Pythona, where we lay our scene,
    From ancient grudge break to new mutiny,
    Where civil code makes git commits unclean.
    From forth the fatal loins of these two foes
    A newer kind of stranger's given life;
    Whose misadventured piteous overthrows
    Doth with its birth bury its parents' strife.

    ACT I

    Enter bpython and ipython

    bpython

    I'm a fancy terminal-based interface to the Python interpreter. I give you
    inline syntax highlighting and auto-completion prompts as you type, and I'll
    even automatically show you a little tooltip with a docstring and parameter
    list as soon as you hit ( to make the function call, so you always know
    what you're doing! I'm svelte and proud of it - I don't try to do all of the
    shenanigans that ipython does with the shell and the web, but the cool kids
    love my rewind feature for demos. I strive to make interactive python coding
    a joy!

    ipython

    I'm an awesome suite of interactive computing ideas that work together.
    For millennia, I've given you tab-completion and object introspection via
    obj? instead of help(obj) in Python. I also have sweet shell features,
    special magic commands (%run, %timeit, %matplotlib, etc.) and a
    history mechanism for both input (command history) and output (results
    caching).

    More recently, I've decoupled the REPL into clients and kernels, allowing
    them to run on independent of each other. One popular client is the
    IPython Notebook which allows you to write code and prose using a web
    browser, sending code to the kernel for execution and getting rich media
    results back inline. The decoupling of clients and kernels also allows
    multiple clients to interact with the same kernel, so you can hook-up to
    that same running kernel from the terminal. The terminal workflow makes
    more sense for some things, but my user interface there isn't as polished
    as bpython's.

    Enter bipython

    bipython

    By your powers combined... I am bipython!

    Exeunt

    The Power is Yours!

    pip install  bipython
    easy_install bipython
    

    bipython requires ipython, pyzmq, bpython, and urwid.

    For now, you'll need to have a running ipython kernel before running bipython. You can do this by either opening a notebook or running ipython console. It won't always be like this, I'll fix it as soon as I can, but it'll be sooner with your help over ivanov/bipython.

    After that, just run bipython and enjoy the ride.

    Here's a walkthrough of ipython, bpython, and bipython:

    The screencast is 20 minutes long, but here I'll play it back double speed. There's no sound, and you can pause at any time and select / copy portion of the text as you like. Changing the browser font size in the usual way works, too. (click here if the embed didn't work)

    permalink
  2. stem-and-leaf plots in a tweet


    Summary: I describe stem plots, how to read them, and how to make them in Python, using 140 characters.

    My friend @JarrodMillman, whose office is across the hall, is teaching a computational statistics course that involves a fair amount programming. He's been grading these homeworks semi-automatically - with python scripts that pull the students' latest changes from GitHub, run some tests, spit out the grade to a JSON file for the student, checks it in and updates a master JSON file that's only accessible to Jarrod. It's been fun periodically tagging along and watching his suite of little programs develop. He came in the other day and said "Do you know of any stem plot implementation in python? I found a few, and I'm using one that's ok, but it looks too complicated."

    For those unfamiliar - a stem plot, or stem-and-leaf plot is a more detailed kind of histogram. On the left you have the stem, which is a prefix to all entries on the right. To the right of the stem, each entry takes up one space just like a bar chart, but still retains information about its actual value.

    So a stem plot of the numbers 31, 41, 59, 26, 53, 58 looks like this:

     2|6
     3|1
     4|1
     5|389
    

    That last line is hard to parse for the un-initiated. There are three entries to the right of the 50 stem, and these three entries 3 8 and 9 is how the numbers 53, 58, and 59 are concisely represented in a stem plot

    As an instructor, you can quickly get a sense of the distribution of grades, without fearing the binning artifact caused by standard histograms. A stem-plot can reveal subtle patterns in the data that are easy to missed with usual grading histograms that have a binwidth of 10. Take this distribution, for example:

    70:XXXXXXX
    80:XXXXXXXXXXX
    90:XXXXXXX
    

    Below are two stem plots which have the same profile as the above, but tell a different story:

     7|7888999
     8|01123477899
     9|3467888
    

    Above is a class that has a rather typical grade distribution that sort of clumps together. But a histogram of the same shape might come from data like this:

     7|0000223
     8|78888999999
     9|0255589
    

    This is a class with 7 students clearly struggling compared to the rest.

    So here's the code for making a stem plot in Python using NumPy. stem() expects an array or list of integers, and prints all stems that span the range of the data provided.

    from __future__ import print_function
    import numpy as np
    def stem(d):
        "A stem-and-leaf plot that fits in a tweet by @ivanov"
        l,t=np.sort(d),10
        O=range(l[0]-l[0]%t,l[-1]+11,t)
        I=np.searchsorted(l,O)
        for e,a,f in zip(I,I[1:],O): print('%3d|'%(f/t),*(l[e:a]-f),sep='')
    

    Yes, it isn't pretty, a fair amount of code golfing went into making this work. It is a good example for the kind of code you should not write, especially since I had a little bit of fun with the variable names using characters that look similar to others, especially in sans-serif typefaces (lI10O). Nevertheless, it's kind of fun to fit much functionality into 140 characters.

    Here's my original tweet:

    You can test it by running it on some generated data:

    >>> data = np.random.poisson(355, 113)
    >>> data
    array([367, 334, 317, 351, 375, 372, 350, 352, 350, 344, 359, 355, 358,
       389, 335, 361, 363, 343, 340, 337, 378, 336, 382, 344, 359, 366,
       368, 327, 364, 365, 347, 328, 331, 358, 370, 346, 325, 332, 387,
       355, 359, 342, 353, 367, 389, 390, 337, 364, 346, 346, 346, 365,
       330, 363, 370, 388, 380, 332, 369, 347, 370, 366, 372, 310, 348,
       355, 408, 349, 326, 334, 355, 329, 363, 337, 330, 355, 367, 333,
       298, 387, 342, 337, 362, 337, 378, 326, 349, 357, 338, 349, 366,
       339, 362, 371, 357, 358, 316, 336, 374, 336, 354, 374, 366, 352,
       374, 339, 336, 354, 338, 348, 366, 370, 333])
    >>> stem(data)
     29|8
     30|
     31|067
     32|566789
     33|00122334456666777778899
     34|02234466667788999
     35|001223445555577888999
     36|12233344556666677789
     37|0000122444588
     38|0277899
     39|0
     40|8
    

    If you prefer to have spaces between entries, take out the sep='' from the last line.

    >>> stem(data)
     29| 8
     30|
     31| 0 6 7
     32| 5 6 6 7 8 9
     33| 0 0 1 2 2 3 3 4 4 5 6 6 6 6 7 7 7 7 7 8 8 9 9
     34| 0 2 2 3 4 4 6 6 6 6 7 7 8 8 9 9 9
     35| 0 0 1 2 2 3 4 4 5 5 5 5 5 7 7 8 8 8 9 9 9
     36| 1 2 2 3 3 3 4 4 5 5 6 6 6 6 6 7 7 7 8 9
     37| 0 0 0 0 1 2 2 4 4 4 5 8 8
     38| 0 2 7 7 8 9 9
     39| 0
     40| 8
    

    To skip over empty stems, add e!=a and in front of print. This will remove the 300 stem from the output (useful for data with lots of gaps).

    >>> stem(data)
     29| 8
     31| 0 6 7
     32| 5 6 6 7 8 9
     33| 0 0 1 2 2 3 3 4 4 5 6 6 6 6 7 7 7 7 7 8 8 9 9
     34| 0 2 2 3 4 4 6 6 6 6 7 7 8 8 9 9 9
     35| 0 0 1 2 2 3 4 4 5 5 5 5 5 7 7 8 8 8 9 9 9
     36| 1 2 2 3 3 3 4 4 5 5 6 6 6 6 6 7 7 7 8 9
     37| 0 0 0 0 1 2 2 4 4 4 5 8 8
     38| 0 2 7 7 8 9 9
     39| 0
     40| 8
    

    Thanks for reading.

    permalink
  3. disabling blinking

    2014 01 29 technology

    python vision

    Background: Text editing in the IPython Notebook is provided by an excellent JavaScript-based CodeMirror text editor. This might be more detail than you want, but I'm a vision scientist so I hope you'll indulge me.

    The cursor is meant to tell the user the current location.

    The human visual system has a pre-cortical lag of roughly 50-90 ms (read more about P1).

    That's how long it takes from something changing on the screen to cause an avalanche of photons to barrel towards your eyeball, be phototransduced and processed by several stages of cells in the retina, finally causing retinal ganglion cells to fire an action potential down their axons through the optic nerve, make its way to a processing relay station called the LGN, with those cells firing action potential down their axons, with those spikes finally ending up in the primary visual cortex.

    By ~150 ms, our brains have processed the visual input enough to perform a non- trivial ammount of object recognition.

    The default blink rate for CodeMirror is 530ms.

    That's as slow as molasses in January!

    Say that I've been distracted and looked away from the screen. When I look back at the scree, half of the time it will take 3 times longer for me to get the information that I want ("where's my cursor") than if that cursor was clearly visible at all times. Now it's not always that bad, because sometimes my gaze will land on the screen and even though the cursor isn't visible, it appears in a few milliseconds, and so it takes just as long as if the cursor was there the whole time. But if I happen to be particularly unlucky (there's a reason I don't gamble), it can take 6 times longer.

    Try it out

    Here's the bit of JavaScript code you need to disable blinking in CodeMirror.

    CodeMirror.defaults.cursorBlinkRate=0
    

    If you type that into the JavaScript console of your webbrowser, that setting will apply to all cells created in the current IPython Notebook. If you don't know how to open your browser's Javascript console, don't frett, just make a new cell with just the following lines in there, execute it, and make a new cell to see how you like it.

    %%javascript
    CodeMirror.defaults.cursorBlinkRate=0
    

    Make the change stick

    IPython has a notion of profiles to allow for different kinds of configurations. If this is news to you, you've probably just been using the default profile and not known it. In the shell, run the ipython profile create command to be sure (don't worry, if you alreay have a profile, this won't overwrite it). Now ipython locate profile will tell you the directory which contains all of the configuration for the default profile.

    In [1]:
    !ipython profile create
    
    In [2]:
    !ipython locate profile
    
    /home/pi/.ipython/profile_default
    
    In [3]:
    x = !ipython locate profile
    
    In [4]:
    cd $x.s
    
    /home/pi/.ipython/profile_default
    
    In [5]:
    ls
    
    db/  history.sqlite  history.sqlite-journal  ipython_config.py  ipython_nbconvert_config.py  ipython_notebook_config.py  log/  pid/  security/  startup/  static/
    
    There's a lot of stuff there, but we just need to add our one line to the end of the file in ``static/custom/custom.js``
    In [6]:
    cd static/custom/
    
    /home/pi/.ipython/profile_default/static/custom
    
    In [7]:
    ls
    
    custom.css  custom.js
    
    In [8]:
    !echo "codemirror.defaults.cursorblinkrate=0" >> custom.js
    
    ## "I want it all and I want it now!" You say you don't want to save your current notebook and reload it to get the updated CodeMirror settings? You just want all cells in the current notebook to change their behavior? Well, OK, Freddie:
    In [9]:
    %%javascript
    var rate = 0;
    // apply setting to  all current CodeMirror instances
    IPython.notebook.get_cells().map(
        function(c) {  return c.code_mirror.options.cursorBlinkRate=rate;  }
    );
    
    // make sure new CodeMirror instance also use this setting
    CodeMirror.defaults.cursorBlinkRate=rate;
    
    I hope you enjoyed this little IPython customization detour. If you want more information about how to get rid of blinking in other programs you use every day, [here is an invaluable resource on that matter](http://www.jurta.org/en/prog/noblink). Remember, blinking in user interfaces is bad, [but blinking in vision is very important](http://www.brower.co.uk/opticians/blinking.html). ** This post was written as an IPython Notebook. You can view it on [nbviewer](http://nbviewer.ipython.org/url/pirsquared.org/blog/notebooks/notebook-blink.ipynb), or [download it](/blog/notebooks/notebook-blinking.ipynb) ** permalink
  4. What happens to a talk deferred...

    2013 06 17 technology

    python talk

    I'll be in Austin, TX for SciPy next week, and I'll be giving an updated talk about tools for reproducible research.

    What follows is the abstract of a talk that I really wanted to give and submitted to the general track which did not get in. I care deeply about this topic, and I'm hoping to have conversations with others about it. The talk was deferred to a poster, so if you don't bump into me at other times, seek me out during the poster session (mine's poster #15, 10:35 AM - 11:35 AM on June 27th). This post is intended as a pre-conference warm up to that.

    Navigating the Scientific Python Communities - the missing guide.

    The awful truth: our newcomers have a deluge of options to wade through as they begin their journey. What tools are available, how do I install them, how do I make them work together -- all hard questions facing a budding scipythonista.

    On developer-friendly platforms, the popular approach is to just install numpy, scipy, matplotlib, and ipython using package management facilities provided by the operating system. On Mac OS X and Windows, the least painful, bootstrapping approach is to use a Python distribution like Python(X,Y), EPD, Anaconda, or Sage.

    Both of these paths obscure a reality which must be stated explicitly: the development of packages is fundamentally decentralized. The scientific python ecosystem consists of a loose but thriving confederation of projects and communities.

    The chaos of installation options and lack of centralization around a canonical solution, which on the surface appears to be a point of weakness that is detrimental to community growth, is a symptom of one of its greatest strengths. Namely, what we have is a direct democracy for user-developers. They vote with their feet: filing bug reports, testing pre-release versions, participating in mailing lists, writing and reviewing pull requests, and advertising the tools they use in talks, papers, and conversations at conferences.

    I will discuss why this is the case, why this is a good thing, and how we can embrace it. The talk will present the ethos and expectations of an effective newcomer, as well as resources and strategies for incremental progress toward becoming a master scipythonista.

    permalink
  5. tags-vs-categories in Pelican

    So, following up on my transition to pelican, here's a little help for navigating a pelican site. If you've ever wondered why sometimes there's a highlighted menu item at the top, and other times there isn't, read on.

    Categories

    With Pelican, every post belongs to exactly one category - that category is 'misc' if one is not provided (and you can override that named by setting DEFAULT_CATEGORY = 'somethingelse' in your pelicanconf.py file). Alternatively, if the post is in a subdirectory, it's category becomes the subdirectory name. So if you have a situation like this:

    .../content/unicorns/lady-rainicorn.md
    

    The lady-rainicorn post will be filed under the category of unicorns. You can explicitly override this category-from-subdirectory behavior, by specifying the category in the header of the file, like so:

    title: All about Lady Rainicorn
    slug: lady-rainicorn
    category: adventure-time
    

    Categories are where a lot of the pelican themes get the top navigation bar, which is referred to as the menu in Pelican. You'll notice that this post is filed under technology. If you're viewing this on the main page, that won't be highlighted, but if you click on the title of this post, you'll see that technology will become highlighted. Same deal if you just click on technology, which will get you a listing of the latest post in that category, and a pagination of all previous ones.

    Pages

    The menu items need not be limited to categories, however. With the popular notmyidea template (which is what I've customized for my journal here), if you create any pages (by putting the file in a content/pages/ directory), then the title of that page will appear in front of all of the categories in the menu. These pages don't show up in RSS feeds, though.

    Menuitems

    Additionally, you can specify other MENUITEMS in pelicanconf.py, which will go in front of all pages and categories. MENUITEMS is a list of tuples, each tuple should be composed of a link name, and a url. This is how I link to the listing of all of my blog posts:

    MENUITEMS = [('all', '/blog/archives.html')]
    

    Tags

    Finally, you can have zero or more tags attached to a post. Tags, by default, do not get a feed generated for them. To enable a tag feed, you'll have to set a line like this in your pelicanconf.py files:

    TAG_FEED_ATOM = "feeds/tag_%s.atom.xml"
    

    Menu highlighting

    The notmyidea template only contains logic for highlighting the active category. This is why, if you click on 'All' at the top of mine, it doesn't stay highlighted when you're on that page.

    On the other hand, if you click at cycling at the menu bar, it will stay highlighted, and you will see a listing of the articles in that category. But if you click on the cycling tag in one of those posts - the menu item will not be highlighted, and you will see some more articles - in particular, right now my review of Just Ride will also be listed, but it lives in the category of books, because that's where I'm keeping book reviews.

    Summary

    So there you have it - now you know the differences between categories, tags, pages, and menu items, as far as Pelican is concerned. For me, the bottom line is that I'll probably stick to just using categories, and will try to use tags sparingly. In my transition from WordPress, I cleaned up and removed a whole bunch of tags (since they were either redundant (the same tags kept appearing together), or were too specific (most tags were only used for one post).

    permalink
  6. embracing my hypertextuality

    Well, it's happened again, I've jumped (back) on the static blog engine bandwagon. Early versions of my site were generated literally using #define ,#include, gcc, and a Makefile...). Back then I was transitioning away from using livejournal, and decided to use WordPress so I wouldn't have to roll my own RSS feed generator.

    I tolerated WP for a while - and the frequency of my posts was so low, that it wasn't much of an issue.

    Except for the security upgrades. I logged into the wp-admin console way more times than I cared to just to press the little "upgrade" box. The reason is that wordpress keeps everything in a database that gets queried every time someone hits the site. I was never comfortable with the fact, because content can be lost in case of database corruption during either an upgrade or a security breech. Also, my content just isn't that dynamic. The WP-cache stuff just seemed overkill, since I don't get that many visitors.

    But lately, I've found myself wanting to write more, to post more, but also shying away from it because I hate dealing with the WordPress editor. And I also hate being uncertain about whether any of it will survive the next upgrade, or the next security hole, whichever I happen to stub my toe on first.

    And the thing is, I really like to use version control for everything I do. I liked my blog posts to be just text files I can check into version control. I also like typing "make" to generate the blog, and now I get to!

    For added fun, I'm hoping that writing my posts in markdown will make it easier to coordinate my gopher presence, since it's pretty close.

    For posterity, I'm capturing what the first version of my Pelican-based blog looked like. I did the same thing when I moved to WordPress.

    Embracing my
hypertextuality

    I ran into some confusing things about transitioning to Pelican, so I thought I'd note them here, for the benefit of others.

    Unadulturated code blocks

    I like to use indentation as a proxy for the venerable <tt> tag - which uses a monospace font.

    If you just want to use an indentation, but do not want the indented text parsed as a programming language, put a :::text at the top of that block. Here's what I mean. Take this Oscar Wilde quote, where I've inserted a line break for drammatic effect, for example:

    A gentleman is one who never hurts anyone's feelings
    unintentionally.
    

    Now, you see that ugly red box around the apostrophe? Well, That's because all I did was indent the two lines. If I just put a :::text above the quote, indented to the same level,

        :::text
        A gentleman is one who never hurts anyone's feelings
        unintentionally.
    

    the result will render like this.

    A gentleman is one who never hurts anyone's feelings
    unintentionally.
    

    As the pelican documentation specifies, this is also the way you can also specify the specific programming language you want, so :::python would be one way to not make pygments guess. You can get a list of all supported languages here, just use one of the short names for your language of choice.

    And you should really do this, even if you aren't bothered by the red marks, because the code highlighting plugin goes in and tokenizes all of those words and surrounds them in <span> tags. Here's the HTML generated for the first version:

    <div class="codehilite"><pre><span class="n">A</span> <span class="n">gentleman</span> <span class="n">is</span> <span class="n">one</span> <span class="n">who</span> <span class="n">never</span> <span class="n">hurts</span> <span class="n">anyone</span><span class="err">&#39;</span><span class="n">s</span> <span class="n">feelings</span>
    <span class="n">unintentionally</span><span class="p">.</span>
    </pre></div>
    

    and here's the version generated when you add the :::text line at the top:

    <div class="codehilite"><pre>A gentleman is one who never hurts anyone&#39;s feelings
    unintentionally.
    </pre></div>
    

    nikola?

    At some point, having a dialogue with myself, I wrote in here "or should I not use pelican and use nicola instead?"

    Ok, tried it - nikola takes too long to startup -

    nikola --help
    real    0m1.202s
    user    0m0.876s
    sys     0m0.300s
    
    pelican --help
    real    0m0.639s
    user    0m0.496s
    sys 0m0.132s
    

    I'm sure it's a fine static blogging engine - and Damian Avila's already written IPython Notebook converters for it, but it just feels like it tries to do too many things. Constraints are good. I'll stick with Pelican for now. (Though I did use the nikola wordpress import tool to grab the wp-upload images from my WordPress blog)

    another set of instructions I consulted: Kevin Deldycke's WordPress to Pelican, which is how I did get my articles out using exitwp which I patched slightly, so files got saved as .md, and preserve other format properties.

    Redirects

    also known as: not breaking the web

    I wanted to preserve rss feeds, and also not break old WordPress style /YYYY/MM/DD urls - the Nikola wp-import script had created a url remapping scheme in a file called url_map.csv.

    I don't have that many posts, so I just added them in by hand:

    Options +FollowSymLinks
    RewriteEngine on
    RedirectMatch 301 /blog/feed/ /blog/feeds/all.atom.xml
    RedirectMatch 301 /blog/2006/10/18/todd-chritien-greens-choice-voting/ /blog/todd-chritien-greens-choice-voting.html
    RedirectMatch 301 /blog/2007/01/04/changelogs-with-dates-gui-goodness/ /blog/changelogs-with-dates-gui-goodness.html
    ...
    

    Enabling table of contents for posts

    If you want to include a table of contents within a post using [TOC], you must enabled the markdown toc processor with a line like this is your pelicanconf.py:

    MD_EXTENSIONS =  [ 'toc', 'codehilite','extra']
    

    Categories and tags

    Ok, so this was never clear to me in wordpress, either - but what's the difference between a tag and a category? is it the case that a post can only belong to one category, whereas it can have any number of tags?

    I think I used categories as tags on wordpress. Looks like all posts on Pelican can have at most one category. Turns out this little aside was long enough to turn into its own post, so if you're interested, pelican tags-vs-categories has got you covered.

    That's it for now

    Thanks to Preston Holmes (@ptone) for encouraging me to transition away from WordPress, and pointing me to this post by Gabe Weatherhead (@MacDrifter) for how to do that. It should be said that the pelican documentation itself is very good for getting you going. Additionally, I consulted this post by Steve George which has a good description to get you started, and also covers a bunch of little gotchas, and lots of pointers. Also, thanks to Jake Vanderplas (@jakevdp) for his writeup on transitioning to Pelican, which I will consult later for incorporating IPython notebooks into my markdown posts, in the future. This is good enough for now. LTS.

    permalink
  7. remembering John Hunter (1968-2012)

    John Hunter, the author of matplotlib, passed away on August 28th, 2012. He will be dearly missed.

    Please donate to the John Hunter Memorial Fund. A giant in our community, John lead by example and gave us all so much. This is one small way we can give back to his family.

    what follows are excerpts from my paper journal over the past week:

    John Hunter passed away this morning,
    Oh my god. John Hunter was incredibly
    kind and warm -- I can't believe he bought
    me this laptop. I can't believe that such
    a wonderful man could be dead -- so suddenly.
    
        What a tremendous loss.
    

    Dear Merlin,
        John Hunter died yesterday --
    getting unstuck. It is entirely appropriate
    to be stuck -- to feel it, smell it, taste it
    feel it drilling though your head
    

    I keep pacing around the house.
    I just need to leave.
        calmly.
    My thoughts are with John's
    family. & with the folks in Louisiana.
    
        A rat in a maze --
    panicked -- & the water
    level rises still. escalates.
    

        Our loss of John makes me
    want to code furiously.
    

    I feel the urge to code furiously,
    but only have the capacity to tweet about
    it, and lack thereof to censor myself.
    

    I don't believe I'm losing
    my mind. I believe I never had
    one to begin with.
      I'm not losing my mind. I never
      had one to begin with.
      I'm not losing my mind.
      That would imply I had one
      to begin with.
    Left the house without shoes --
    waiting for my laptop to charge.
    
      I want to share this with the
      group -- so I can come to
    terms with it myself. forgive myself.
    
      Connection with a stranger -- can be a form of escape.
    but it can help you gain perspective on
    your own life. I think it has for
    me.
      I am a severely broken
    person. Ok. Time to get shoes,
    shower, & then go to try &
    talk w/ Greg Wilson.
    

    I must be mistaken. Maybe I was.
    Who's to say that I wasn't.
    

    The scientific python community lost one
    of it's giants this week, and
    I lost an important mentor.
    
    Remembering John Hunter (JDH)
    
        People keep dying. I don't know
    how to deal with that -- I feel like
    I never really processed dedushka's death --
    nor Ken Green's, nor Jessi Debaca's,
    nor babushka's
    
    ... I lost another mentor. Most of
    these people (all?) don't know that they
    were mentors to me -- but they were.
    I looked up to John -- I secretly
    wanted to please him -- but did not
    even dare to do so directly (I
    have Fernando Perez to thank for getting
    my first contribution into Matplotlib.
    
    I remember feeling really bad after my first
    sprints at SciPy 2009 -- John actually
    knew who I was -- and wanted me
    to work on some matplotlib stuff --
      I ended up doing some rote work with
    David Warde Farley (and felt kind
    of like a third wheel - since dwf
    (pronounced "dwoof" - did I get
    that right, David?) is more than
    capable as a command-line
    cowboy (David doesn't know this --
    but I learned a great deal from
    just sitting next to him and watching
    him string together standard unix
    tools, pipe-after-pipe -- to clean
    up some scipy wiki content, to
    try and export it to a new site.
    David's also a role model in other ways --
    he's very calm and collected (unless
    he senses you've pushed the bozo button from
    some punditry vending machine - instead of
    understanding and engaging with the full complexity of some
    social or political issue)
    
        John was extremely kind and understanding -- 
    he wanted to invite & welcome me to
    code alongside other matplotlib developers,
    but there was no expectation.
    
      I got to hang out with John the 
    most at the PyData conference in
    Mountain View in late February. He was
    giving a matplotlib talk, and was seeking
    feedback on what to talk about --
    & how to do it. He's warm -- and always
    had this kind smile about him.
    
        After hanging out with John at PyData --
    this email arrived in my inbox
    

    Hey Paul, It was great seeing you again at pydata, and thanks for your help during the talk. We’ve decided that you could be a lot more productive in all your efforts to help ipython, mpl and others if you had a shiny new laptop, so I want to buy you one out of the mpl donations fund. If you spec out the machine you want and send me a link and details, and your preferred shipping address, I’ll order it for you and have it shipped to you. Pick a machine good enough that you can rely on it for a few years, because it looks like you get good use from these things! In other words, don’t feel compelled to be frugal on our behalf.  JDH

    Fernnado knew about the email -- & I was at
    a barbecue at his house -- to sort of
    celebrate an awesome week of lots of
    Python Giants in from out of town.
      Fernando was giddy -- "Have
    you checked your email?" -- with a
    twinkle in his eye "Let's go have you
    check your email right now" -- & he
    walked over to turn on his desktop machine.
    

        On the mailing lists -- I always tried to
    emulate John's approach -- helping everyone
    even if in the slightest manner.
    
    I was offered commit rights sort
    of out of the blue -- after sending
    a couple of pull requests.
    
        We are a community.
    We need to remember John, & keep
    remembering John -- for many, many,
    many years to come.
    
    I feel the urge to reach
    out to everyone I know & don't know.
    I'm desperate -- i think I've been that
    way since I was a little kid -- I
    remember having the same feelings when
    we were leaving Moscow -- a ten-year old,
    talking to friends and my teachers,
    many for the very last time.
    

    I am posting this, but it isn't finished.
    
    This is broken, half-finished, confused,
    necessarily so -- because there is no way
    to mend what we have lost. This
    is an expression of my current state --
    and if I don't let this out, it will just
    keep ricocheting around in my head for years
    to come.
    
        This is a first pass. This is me
    grieving. This will never be enough. This
    is just a start.
    

    videos of John Hunter's talks:

    matplotlib: Lessons from middle age. (Scipy 2012)

    Advanced Matplotlib Tutorial (PyData)

    SciPy 2009 - Advanced tutorial 3: Advanced topics in matplotlib

    (beginning) SampleDoc

    Scipy '09 Panel on Visualization tools

    Scipy '02 Core Projects update

    NIPS Workshop on Machine Learning Open Source Software (MLOSS)

    Fernando Perez and John Hunter

    Sept 2009: "some fun stories like 'Jeez, you guys have some crazy examples. I am surprised there isn't dolphins swimming around inside a sphere.' So now there is."

    permalink
  8. pheriday 2: termcasting overview

    pheriday 2: termcasting overview (2012-08-03) from Paul Ivanov on Vimeo.

    paul's habitual errant ramblings (on Fr)idays (2012-08-03)

    show notes:
    http://pirsquared.org/blog/2012/08/04/termcasting/
    gopher://sdf.org/1/users/ivanov/pheridays/2012-08-03 (yes, gopher!)

    1. try to not say "uuuuhhhhmmnn"

    BAM/PFA Summer Cinema on Center Street http://bampfa.berkeley.edu/filmseries/summercinema

    1. SciPy 2012 videos up, go check them out! (I have!) http://www.youtube.com/nextdayvideo (removed nextdayvideo internal box url, by request)

    Software Carpentry: Record and Playback post http://software-carpentry.org/2012/07/record-and-playback/

    1. termcasting: a review of what's out there.

    http://termcast.org and http://alt.org/nethack/ mostly nethack stuff, both just use telnet protocol, only live sessions (though there are "TV" scripts to re-run sets of ttyrec files).

    (playterm vs ascii.io vs shelr.tv)

    tldr: playterm.org supports ttyrec files, but has the most primitive player. Players on ascii.io and shelr.tv can both seek. shelr.tv can also speed up playback! Downside is both of those have their own recorder programs (though at least shelr leverages script or ttyrec)

    http://playterm.org/ - supports ttyrec files, outgoing links for author and related article, comments. - most primitive player (https://github.com/encryptio/jsttyplay/) - Pause only - only terminal sized of 80x24 or 120x35 - supports tags and comments - service only (code for playterm.org does not seem to be available, though jsttyplay is doing the hardest part of actual playback)

    http://ascii.io/ - supports non-standard terminal size - player can seek. - aesthetic thumbnail previews - login via github or twitter credentials (for uploads) - code for website available (ruby and javascript) https://github.com/sickill/ascii.io - code for recorder available (python) https://github.com/sickill/ascii.io-cli

    http://shelr.tv/ - supports non-standard terminal size - player can seek. - player playback speed can be increased (currently up to 10x of real time) - supports tags, comments and voting up/down on a video - shelr can playback from command line ("shelr play http://shelr.tv/records/4f8f30389660802671000012.json") - code for website available (ruby and javascript) [AGPLv3] https://github.com/shelr/shelr.tv - code for recorder available (ruby) [GPLv3] https://github.com/shelr/shelr

    1. my wanted list for termcasting
    2. should support ttyrec files (upload and download)
    3. live-streaming (like ustream - but for coding)
    4. termcast.org has ttrtail which does just this
    5. quick "encrypt" switch - to keep streaming, but start GPG encrypting the stream as it goes out - so you can still look at it later. This would make it easy to leave the streaming on all the time
    6. a .tty editor that's like a video editor cut out portions [i.e. dead time]

    This is a low-bandwidth way of capturing what I'm working on and thinking about. Now, I'm going to try to record everything I do! "ttyrec -e screen -x". I've only done it a couple of times so far while coding, but I find being able to go back and re-view (and review) what I worked on at the end of the day to be really helpful.

    I was inspired by Joey Hess' "git-annex coding in haskell" where he reviews and narrates some of the code he wrote, after he wrote it. http://joeyh.name/screencasts/git-annex_coding_in_haskell/

    P.S. It's Saturday now. I tried to save some local diskspace by running recordmydesktop using the --on-the-fly-encoding option, and that was a mistake. The audio and video were (un)hilariously desynchronized - the audio ran for 9:48, but the video wanted to be just 7:30. Audacity came to the rescue by allowing me to change the tempo to be 30% faster, which made the syncing better. And then I used avconv to stitch in the faster audio.

    tools used: Debian GNU/Linux sid, recordmydesktop, xmonad, fbpanel, screen, chromium, cheese, xcompmgr, audacity, avconv

    permalink
  9. pheriday 1: software carpentry, digital artifacts, visiting other OSes

    Here's pheriday 1, another edition of paul's habitual errant ramblings (on Fr)idays

    pheriday 1: software carpentry, digital artifacts, visiting other OSes (2012-07-27) from Paul Ivanov on Vimeo.

    2012-07-27.mp4 (28 MB) 2012-07-27.avi (71 MB) 2012-07-27.ogv (321 MB)

    show notes

    I had three topics I wanted to cover today, and ended up spending about an hour thinking about what I was going to say and which resources I was going to include. This was too long, and the end result was still very rambling, but I think I'll get better at this with more practice.

    SDF Public Access UNIX System http://sdf.org gopher://sdf.org/1

    1. try to not say "uuuuhhhhmmnn"
    2. be lazy (software carpentry)
    3. flipside of "publisher's block" (git-annex)
    4. visiting another country (windows 8 release preview)

    As usual, I didn't know what I was really trying to say in 0, and here's a really good overview of what I meant: Software Carpentry

    If you have 90 seconds, watch the pitch

    play it in 60 seconds, instead: mplayer -af scaletempo -speed 1.5 Software_Carpentry_in_90_Seconds-AHt3mgViyCs.flv

    1. publisher's block

    Jaron Lanier's You Are Not A Gadget What I mention in the video is not at all the main point of Lanier's book (which is quite good!), and in fact, his book is a critique of (over) digitization. Nevertheless, I'm only pointing out that there are redeemable aspects of an increasingly digital artifact producing life, such as preservation.

    David Weinberger’s Everything is Miscellaneous

    My review of David Weinberger’s Everything is Miscellaneous, where I go into more depth about "information overload".

    git-annex Excellent project. The technical details is that when you "annex" files, they are renamed to long hash of their contents (bit rot resistant!) and stored in a .git/annex/objects directory, whereas in place of where the file was, you get a symlink to the original file, which gets added to git. So git only keeps track of symlinks, and additionally has a git-annex branch that keeps track of all known annexes, so that you can copy, move, and drop files from the ones that are accessible. Very handy!

    Haiku OS

    tools used: Debian GNU/Linux sid, recordmydesktop, xmonad, fbpanel, screen, iceweasel, cheese, xcompmgr, youtube-dl, mplayer, screen

    gopher version of this post (proxy)

    permalink
  10. pheriday 0: scientist-hacker howto (video post)

    Hey everyone, here's pheriday 0, the first of paul's habitual errant ramblings (on Fr)idays

    pheriday 0: scientist-hacker howto (2012-07-20) from Paul Ivanov on Vimeo.

    Berkeley Kite Festival (510 Families)

    Merlin Mann's Most Days (specifically the travel day one on 2009-01-11)

    Sad that I missed SciPy Conference this year. One of the things I like doing at scipy is nerding it up with my friends, seeing each others workflows, showing off vim tricks, etc. This video was my attempt at scratching that itch, a little bit. As I mention in the video, this is take 2. Take 1 ended when I ran out disk space, but needless to say, it was more awesome than this. It seems I am cursed with losing first takes, see also a summary of last year's SciPy conference, where this exact same thing happened.

    NumFOCUS: NumPy Foundation for Open Code for Usable Science

    NumFOCUS Google Group see thread titled: "[Funding] Notes from Funding BOF at SciPy2012"

    TLDP: The Linux Documentation Project (page I was scrolling through)

    Transition to Gopher was rough this time, it was better during the first take.

    Lorance Stinson's w3m (better) gopher support Use this if, for example, going to w3m gopher://sdf.org you get errors like:

    [unsupported] '/1' doesn't exist! [unsupported] This resource cannot be located.
    

    It still took some tweaking, shoot me an email for details

    Robert Bigelow's About | Gopher & GopherSpace

    Here's the HTTP Proxied version of the above: Gopher proxy provided by Floodgap

    SDF Public Access UNIX System http://sdf.org gopher://sdf.org/1

    Eric S. Raymond's How To Become a Hacker Howto

    Fernando Perez' Py4Science Starter Kit

    Q: Why are you using "Chromium --incognito"? I have chronic tabitis, and this is one way of mitigating that problem. If the browser crashes or I shutdown my computer, I won't have those tabs around anymore.

    programs used: Debian GNU/Linux sid, recordmydesktop, xmonad, fbpanel, screen, chromium, cheese, xcompmgr, mutt, wyrd, tail, w3m

    gopher version of this post (proxy)

    permalink

« Page 2 / 3 »