Other articles


  1. pheriday 3: infrastructure

    paul's habitual errant ramblings (on Fr)idays

    pheridays: 3

    2020-04-10: A week ago, I recorded a 5 minute audio segment of some stuff I've been thinking about, but when I started to write it up I stumbled into and kept dropping down a deep technostalgic hole.

    fall down along with me:

    https://pirsquared.org/blog/pheriday-infrastructure.html

    The recording is just shy of five minutes long, you can also download it in different formats, depending on your needs, if the audio tag above doesn't suit you:

    https://pirsquared.org/pheridays/2020-04-03.ogg (2.9 Mb)
    https://pirsquared.org/pheridays/2020-04-03.mp3 (4.5 Mb)
    https://pirsquared.org/pheridays/2020-04-03.m4a (6.3 Mb)

    --

    Stuff I mentioned in the audio:

    Propellor - "configuration management system using Haskell and Git" by Joey Hess

    OpenWRT - specifically - reducing Bufferbloat

    Mumble - "a free, open source, low latency, high quality voice chat application."

    sourcehut.org - "the hacker's forge" also know as sr.ht by Drew DeVault

    Jitsi - "Multi-platform open-source video conferencing"

    OpenFire - "real time collaboration (RTC) server licensed under the Open Source Apache License." Extensible XMPP server, with plugins, like a Jitsi-based video meeeting one claled OpenFire Meetings.


    Though this is the fourth installment, the last time I recorded and posted a rambling was back almost 8 years ago! In fact, it was 2012-08-03, so 7 years and 8 months, to the day.


    Having control of your infrastructure is a longtime thread for me.

    For starters - there's the bicycle. That's been my primary and preferred mode of transportation for 30 years. As a kid, I was empowered by the sense of freedom, independence, and self-sufficiency that came with a bike. All these years later, I'm still a fan. You can see just how happy I am on a bike at the top of this interview , thanks to a sweet photo that was taken by Robert Sexton right by the Golden Gate Bridge at the end of the Lucas Valley Populaire in 2015.

    Those of you who knew me back in college might remember how at UC Davis I ran my own "pirate" internet radio station - KPVL - with the cheeky tagline of "More broadcasters than listeners". (I say "pirate" because it has not relation to the actual KPVL radio station). But there are earlier remnants and traces of my efforts to exercise control and build my own reality.

    I think it was in 1999 that my brother Mike and I started using Redhat (6), then Mandrake Linux 6.5, dual booting on a computer at home and I separately around the same time I got myself an sdf.org account. Though I wasn't sophisticated enough to have a constant internet connection in high school, I was lucky enough to get an account on Robert Chin's laya.com server. The url was - p.laya.com - it's long gone, but luckily, Archive.org has a copy from 2001.

    Wow. I just took a look and so much came flooding back.

    Here's the thing: April is an anniversary of sorts for me. Back in 1999, it marks my first time breaking anonymity and pseudonimity and using my real name on the internet. I've written about this before under the title of Publisher's block ten years ago - just about half way between now and then. This time, though, let me inline the piece I linked to as proof of the deliberate nature of my lack of anonymity.

    An account of my life at 15, as I live it.
    traces of my awareness of the world, I can look back at later
    My first attempt at a memoir
    My goal is to capture my many thoughts emotions, behaviors, incidents, and
    acquaintances
    and to arrive only at an exponential number of those,
    hoping yet being afraid that it might be zero
    making everything about me: one
    

    I'm glad I can now reflect on the kind of kid I was, thanks to the amazing folks who had the foresight to start archiving all of the web for The Way Back Machine. I used that p.laya.com page as a todo list and notes for myself using a hipster combination of the default file index listing with a FOOTER.html. It was captured in 2001, I was in 17, but some of this was written when I was 15 or 16 (I found contents from November 2000), I make mention to my then freeshell.org account (it's now been ivanov@ since 2012). I link to the source code of a MUD - ftp://ftp.game.org/pub/mud/diku/merc/rom/tartarus/tartarus.tgz - which is a broken link now, but I found a mirror over here: which is amazing, because just a week or two ago, I was hanging out with fellow SciPy 2020 program co-chairs Madicken Munk and Gil Forsyth over video chat after one of our meetings and I was happily reporting about how one of the positive things to come out of the shelter in place for me is that "I've fixed my mutt configuration and started using it again!" - but they both heard "mutt" as "MUD" and got very excited by that prospect. So much so that we all agreed that we'll have to follow up and actually follow through to build a MUD. And I brought up how at some point in high school I was mildly active in a pair of MUDs, and wanted to make my own, but never got around to it.

    The last link I left for myself on there points to pinkmonkey.com - a homeschooling resource - which is probably handy for the parents with little ones these days.

    Here's the most concrete infrastructure project I can find from then: I collected bookmarks from my friends to share them. The "service" lived at http://p.laya.com/bookmarks - and predates del.icio.us and pinboard. I bet I "advertised" it in my AIM profile.

    If you're curious, there's a link to the archive.org copy near the end of this post, but I had this urge to show it to you much closer to its original glory.

    Let me set the scene: It's Friday in April, the year is 2020, I'm running Windows 10 on my work laptop in poorly connected home in California, where a pandemic has most of the state's residents staying put at home for the several weeks already, and I decided to make a screenshot using the tool du jour of yesteryear

    Netscape Navigator!

    The timestamp on my bookmark website says I last updated it on: Thu Aug 30 20:14:14 PDT 2001

    OldVersion.com tells me that the latest release for Windows that Netscape 4.79 was released in November of that year, and the closest antecedent version available is 4.72 (from February 2000).

    I downloaded it and tried fiddling around with the compatibility settings, but without any luck.

    Then I tried 4.79, and nope, that didn't work, either. So then I tried Netscape 6.01 - release February 2001.

    I happened to have Chrome running at the time because in Firefox I have 1500 tabs open -- fifteen hundred and seven! ;) -- whereas in Chrome it's under 500, so I was trying to tread lightly. How do I know these numbers? For Chrome I found an extension that allows me to copy into the clipboard all open tabs' urls as plain text. It helpfully announces how many such tabs were copied. In Firefox one of the webextension examples gives you a counter.

    Do you remember the web without tabs? Time was, you wanted to visit another webpage, you got two option: you navigate away from whatever you're looking at now, or you hit Ctrl-N to make an new window. I think most people used one or a few windows. But you were not gonna be crazy and open more than a dozen windows. I would have, and probably tried but I couldn't. And session saving across crashes or clean exits? Forget it! That what your history and bookmarks are for, grasshopper.

    But let's get back to the task at hand: this was the lower right of my screen...

    Downloaded Netscape601.exe

    and I decide to start taking screenshots of this journey, click it, and let Windows 10 apply the compatibility settings, and then I'm faced with

    the most improbable error message:

    Setup detected another instance of Netscape 6 is currently running

    :)

    WAT?!

    I didn't think Chrome had any ancestry shared with the Mosaic super-tree, but whatever - you can't exactly argue with software from 2001, and I have an important screenshot to take...

    So now I've quit Chrome, just in case, and going to retry....

    Setup detected another instance of Netscape 6 is currently running

    no dice....

    Damn, what could it be...I've got the Bloomberg Terminal open, I know portions of it are built on Chromium browser technology (had to look it up if this was officially stated somewhere - it is). Ok, so maybe that's what causing the false positive? I close that, and...

    Setup detected another instance of Netscape 6 is currently running

    ...

    I hardly have anything open anymore ...is it VLC?

    ...

    Setup detected another instance of Netscape 6 is currently running

    nope... Ok, what's left still open... Snipping tool I'm using to capture this epic adventure, a few WSL Debian console windows... the voice recorder that started this post... Task manager and Sysinternals' Process Explorer - where I was checking if perhaps somehow the failed attempt at running what was probably a 16 bit version of Navigator 4.72 was still lingering somewhere... SumatraPDF, Windows Terminal (Preview), gVim, and ...

    Zotero?!?

    BINGO!!!!!!!!!

    Initial Netscape 6 Setup screen

    Initial Netscape 6 Setup screen

    Oh right - I guess Zotero uses XUL technology. I didn't really think much about it, but Zotero did start off life as a Firefox extension, and the standalone version came out later, makes sense that it would have grabbed a browser when it struck out on its own.

    At this point I had already sent Madicken, who works at NCSA where Mosaic, the progenitor of Netscape hails from, the first two images... So I wanted to play with fire a bit....

    Now that I've closed Zotero - can I have Firefox 74 open while installing Netscape 6.01?

    Setup detected another instance of Netscape 6 is currently running

    Rats! same error...

    how about Chrome again?

    Initial Netscape 6 Setup screen

    Initial Netscape 6 Setup screen

    oh yeah! Sweet. The world makes sense again.

    Back to the setup.exe...

    I scroll through the EULA - and randomly stop on this section:

    12. HIGH RISK ACTIVITIES.  The Product is not
    fault-tolerant and is not designed, manufactured or
    intended for use or resale as on-line control
    equipment in hazardous environments requiring
    fail-safe performance, such as in the operation of
    nuclear facilities, aircraft navigation or
    communication systems, air traffic control, direct
    life support machines, or weapons systems, in which
    the failure of the Product could lead directly to
    death, personal injury, or severe physical or
    environmental damage ("High Risk Activities").
    

    By the way - here's a good idea I came across a few months ago: throw EULAs (End User License Agreements) into some publicly indexed version control repo (I saw folks using gists for just that sort of thing: here's the Netscape 6.01 EULA.txt)

    Let the folks at Redmond host it.

    Fine. I click next...

    Interesting - there's a "Read Me" button... I do as I'm told, so I click it.

    Oh, a README.txt popped up in Notepad.exe - full contents of that are in that same gist as the EULA

    but in them, there's a link to the full release notes over at

    http://home.netscape.com/eng/mozilla …

    read more
  2. starting my job search

    I am starting to look for a job in the San Francisco Bay Area.

    Since many recruiters ask for and presumably look at GitHub profiles, I decided to give mine a little facelift:

    Smart and Gets Things Done Github Contribution
Graph:

    In case you aren't familiar, that banner was motivated by Joel Spolsky's Smart and Gets Things Done, which is a book about hiring good developers . So I decided to tweet it out, mentioning @spolsky and he favorited it!

    Yesterday, I decided to tweet out an image that's at the top of my resume as a standalone tweet- mentioning Joel Spolsky again, and he liked it well enough to retweet it to his 90 thousand followers, so it's been getting plenty of love.

    Paul Ivanov's Visual Resume

    Perhaps unsurprisingly, the only person to contact me as a result of this so far is a reporter from Business Insider :

    My editor would like to post it on our site as an example of a creative way to format a resume... I'm wondering if we can get your permission to do this?

    So that's what prompted this post: I simply added my name and a Creative Commons Attribution Licence (CC-BY) to the two images, and then sent my permission along.

    Outside of that, no prospective employers have gotten in touch. But like I always say: you can't win the lottery if you don't buy a ticket. And since I also enjoy mixing metaphors, I'll just keep on fishing!

    read more
  3. bipython 0.1.0

    bipython
logo

    the boldly indiscriminate python interpreter

    "...because you shouldn't have to choose."

    PROLOGUE

    Two interpreters, both alike in dignity,
    In fair Pythona, where we lay our scene,
    From ancient grudge break to new mutiny,
    Where civil code makes git commits unclean.
    From forth the fatal loins of these two foes
    A newer kind of stranger's given life;
    Whose misadventured piteous overthrows
    Doth with its birth bury its parents' strife.

    ACT I

    Enter bpython and ipython

    bpython

    I'm a fancy terminal-based interface to the Python interpreter. I give you
    inline syntax highlighting and auto-completion prompts as you type, and I'll
    even automatically show you a little tooltip with a docstring and parameter
    list as soon as you hit ( to make the function call, so you always know
    what you're doing! I'm svelte and proud of it - I don't try to do all of the
    shenanigans that ipython does with the shell and the web, but the cool kids
    love my rewind feature for demos. I strive to make interactive python coding
    a joy!

    ipython

    I'm an awesome suite of interactive computing ideas that work together.
    For millennia, I've given you tab-completion and object introspection via
    obj? instead of help(obj) in Python. I also have sweet shell features,
    special magic commands (%run, %timeit, %matplotlib, etc.) and a
    history mechanism for both input (command history) and output (results
    caching).

    More recently, I've decoupled the REPL into clients and kernels, allowing
    them to run on independent of each other. One popular client is the
    IPython Notebook which allows you to write code and prose using a web
    browser, sending code to the kernel for execution and getting rich media
    results back inline. The decoupling of clients and kernels also allows
    multiple clients to interact with the same kernel, so you can hook-up to
    that same running kernel from the terminal. The terminal workflow makes
    more sense for some things, but my user interface there isn't as polished
    as bpython's.

    Enter bipython

    bipython

    By your powers combined... I am bipython!

    Exeunt

    The Power is Yours!

    pip install  bipython
    easy_install bipython
    

    bipython requires ipython, pyzmq, bpython, and urwid.

    For now, you'll need to have a running ipython kernel before running bipython. You can do this by either opening a notebook or running ipython console. It won't always be like this, I'll fix it as soon as I can, but it'll be sooner with your help over ivanov/bipython.

    After that, just run bipython and enjoy the ride.

    Here's a walkthrough of ipython, bpython, and bipython:

    The screencast is 20 minutes long, but here I'll play it back double speed. There's no sound, and you can pause at any time and select / copy portion of the text as you like. Changing the browser font size in the usual way works, too. (click here if the embed didn't work)

    read more
  4. embracing my hypertextuality

    Well, it's happened again, I've jumped (back) on the static blog engine bandwagon. Early versions of my site were generated literally using #define ,#include, gcc, and a Makefile...). Back then I was transitioning away from using livejournal, and decided to use WordPress so I wouldn't have to roll my own RSS feed generator.

    I tolerated WP for a while - and the frequency of my posts was so low, that it wasn't much of an issue.

    Except for the security upgrades. I logged into the wp-admin console way more times than I cared to just to press the little "upgrade" box. The reason is that wordpress keeps everything in a database that gets queried every time someone hits the site. I was never comfortable with the fact, because content can be lost in case of database corruption during either an upgrade or a security breech. Also, my content just isn't that dynamic. The WP-cache stuff just seemed overkill, since I don't get that many visitors.

    But lately, I've found myself wanting to write more, to post more, but also shying away from it because I hate dealing with the WordPress editor. And I also hate being uncertain about whether any of it will survive the next upgrade, or the next security hole, whichever I happen to stub my toe on first.

    And the thing is, I really like to use version control for everything I do. I liked my blog posts to be just text files I can check into version control. I also like typing "make" to generate the blog, and now I get to!

    For added fun, I'm hoping that writing my posts in markdown will make it easier to coordinate my gopher presence, since it's pretty close.

    For posterity, I'm capturing what the first version of my Pelican-based blog looked like. I did the same thing when I moved to WordPress.

    Embracing my
hypertextuality

    I ran into some confusing things about transitioning to Pelican, so I thought I'd note them here, for the benefit of others.

    Unadulturated code blocks

    I like to use indentation as a proxy for the venerable <tt> tag - which uses a monospace font.

    If you just want to use an indentation, but do not want the indented text parsed as a programming language, put a :::text at the top of that block. Here's what I mean. Take this Oscar Wilde quote, where I've inserted a line break for drammatic effect, for example:

    A gentleman is one who never hurts anyone's feelings
    unintentionally.
    

    Now, you see that ugly red box around the apostrophe? Well, That's because all I did was indent the two lines. If I just put a :::text above the quote, indented to the same level,

        :::text
        A gentleman is one who never hurts anyone's feelings
        unintentionally.
    

    the result will render like this.

    A gentleman is one who never hurts anyone's feelings
    unintentionally.
    

    As the pelican documentation specifies, this is also the way you can also specify the specific programming language you want, so :::python would be one way to not make pygments guess. You can get a list of all supported languages here, just use one of the short names for your language of choice.

    And you should really do this, even if you aren't bothered by the red marks, because the code highlighting plugin goes in and tokenizes all of those words and surrounds them in <span> tags. Here's the HTML generated for the first version:

    <div class="codehilite"><pre><span class="n">A</span> <span class="n">gentleman</span> <span class="n">is</span> <span class="n">one</span> <span class="n">who</span> <span class="n">never</span> <span class="n">hurts</span> <span class="n">anyone</span><span class="err">&#39;</span><span class="n">s</span> <span class="n">feelings</span>
    <span class="n">unintentionally</span><span class="p">.</span>
    </pre></div>
    

    and here's the version generated when you add the :::text line at the top:

    <div class="codehilite"><pre>A gentleman is one who never hurts anyone&#39;s feelings
    unintentionally.
    </pre></div>
    

    nikola?

    At some point, having a dialogue with myself, I wrote in here "or should I not use pelican and use nicola instead?"

    Ok, tried it - nikola takes too long to startup -

    nikola --help
    real    0m1.202s
    user    0m0.876s
    sys     0m0.300s
    
    pelican --help
    real    0m0.639s
    user    0m0.496s
    sys 0m0.132s
    

    I'm sure it's a fine static blogging engine - and Damian Avila's already written IPython Notebook converters for it, but it just feels like it tries to do too many things. Constraints are good. I'll stick with Pelican for now. (Though I did use the nikola wordpress import tool to grab the wp-upload images from my WordPress blog)

    another set of instructions I consulted: Kevin Deldycke's WordPress to Pelican, which is how I did get my articles out using exitwp which I patched slightly, so files got saved as .md, and preserve other format properties.

    Redirects

    also known as: not breaking the web

    I wanted to preserve rss feeds, and also not break old WordPress style /YYYY/MM/DD urls - the Nikola wp-import script had created a url remapping scheme in a file called url_map.csv.

    I don't have that many posts, so I just added them in by hand:

    Options +FollowSymLinks
    RewriteEngine on
    RedirectMatch 301 /blog/feed/ /blog/feeds/all.atom.xml
    RedirectMatch 301 /blog/2006/10/18/todd-chritien-greens-choice-voting/ /blog/todd-chritien-greens-choice-voting.html
    RedirectMatch 301 /blog/2007/01/04/changelogs-with-dates-gui-goodness/ /blog/changelogs-with-dates-gui-goodness.html
    ...
    

    Enabling table of contents for posts

    If you want to include a table of contents within a post using [TOC], you must enabled the markdown toc processor with a line like this is your pelicanconf.py:

    MD_EXTENSIONS =  [ 'toc', 'codehilite','extra']
    

    Categories and tags

    Ok, so this was never clear to me in wordpress, either - but what's the difference between a tag and a category? is it the case that a post can only belong to one category, whereas it can have any number of tags?

    I think I used categories as tags on wordpress. Looks like all posts on Pelican can have at most one category. Turns out this little aside was long enough to turn into its own post, so if you're interested, pelican tags-vs-categories has got you covered.

    That's it for now

    Thanks to Preston Holmes (@ptone) for encouraging me to transition away from WordPress, and pointing me to this post by Gabe Weatherhead (@MacDrifter) for how to do that. It should be said that the pelican documentation itself is very good for getting you going. Additionally, I consulted this post by Steve George which has a good description to get you started, and also covers a bunch of little gotchas, and lots of pointers. Also, thanks to Jake Vanderplas (@jakevdp) for his writeup on transitioning to Pelican, which I will consult later for incorporating IPython notebooks into my markdown posts, in the future. This is good enough for now. LTS.

    read more
  5. pheriday 0: scientist-hacker howto (video post)

    Hey everyone, here's pheriday 0, the first of paul's habitual errant ramblings (on Fr)idays

    pheriday 0: scientist-hacker howto (2012-07-20) from Paul Ivanov on Vimeo.

    Berkeley Kite Festival (510 Families)

    Merlin Mann's Most Days (specifically the travel day one on 2009-01-11)

    Sad that I missed SciPy Conference this year. One of the things I like doing at scipy is nerding it up with my friends, seeing each others workflows, showing off vim tricks, etc. This video was my attempt at scratching that itch, a little bit. As I mention in the video, this is take 2. Take 1 ended when I ran out disk space, but needless to say, it was more awesome than this. It seems I am cursed with losing first takes, see also a summary of last year's SciPy conference, where this exact same thing happened.

    NumFOCUS: NumPy Foundation for Open Code for Usable Science

    NumFOCUS Google Group see thread titled: "[Funding] Notes from Funding BOF at SciPy2012"

    TLDP: The Linux Documentation Project (page I was scrolling through)

    Transition to Gopher was rough this time, it was better during the first take.

    Lorance Stinson's w3m (better) gopher support Use this if, for example, going to w3m gopher://sdf.org you get errors like:

    [unsupported] '/1' doesn't exist! [unsupported] This resource cannot be located.
    

    It still took some tweaking, shoot me an email for details

    Robert Bigelow's About | Gopher & GopherSpace

    Here's the HTTP Proxied version of the above: Gopher proxy provided by Floodgap

    SDF Public Access UNIX System http://sdf.org gopher://sdf.org/1

    Eric S. Raymond's How To Become a Hacker Howto

    Fernando Perez' Py4Science Starter Kit

    Q: Why are you using "Chromium --incognito"? I have chronic tabitis, and this is one way of mitigating that problem. If the browser crashes or I shutdown my computer, I won't have those tabs around anymore.

    programs used: Debian GNU/Linux sid, recordmydesktop, xmonad, fbpanel, screen, chromium, cheese, xcompmgr, mutt, wyrd, tail, w3m

    gopher version of this post (proxy)

    read more
  6. vim-ipython two-way integration! (updated: 2011-08-02)

    I'm very pleased to share with you a demo the forthcoming vim-ipython integration which will work with IPython 0.11(trunk).

    You can either use the Flash player below, or download the OggVorbis file (14MB) update: vim-ipython 'shell' demo (9.6MB). The blog-free form of this post is here.

    If you like what you see and want to try it, you can get the details from the vim-ipython github page and it currently requires 4 line changes to IPython, which are currently in this pull request. (Fixed to work on IPython trunk with no changes).

    Big thanks to Min for walking me through the new IPython kernel manager during the SciPy2011 sprints.

    UPDATE: 2011-08-02

    vim-ipython ‘shell’ mode.

    Just in case, here are the same videos as above, but hosted on Youtube:

    If you're have any issues, try searching for your error on the vim-ipython github issues page, and if you don't find it, please file a new one, and I'll help you out there.

    read more
  7. And it begins again...

    2007 07 20 technology

    hello-world

    screen cap

    So I finally bit the bullet and put up my own blog. It was just one of those wait and see things for a while, but now I find myself reading most things via rss feeds, so I really had no excuse not to move on from my livejournal. I was afraid of abandoning my lj-friends - but Yuan found a happy medium with cross-posting back to her lj (though now that she has an rss feed I read her entries first in Thunderbird, sometimes days ahead of visiting my friends page)

    I'm still getting settled in, so this isn't quite live yet.

    Anyway, I've had a couple of entries on the back burner that I've been working on, and they feel serious enough to warrant having their own place, instead of being a part of a corpus I started almost six years ago (in high school, no less). More and more people I know host their own blogs and it's always nice to have a fresh start (though I've reposted a hand full of my most recent LJ entries to get a running start).

    read more
  8. coming to you live from my desk...

    2007 01 22 life

    hello-world

    So I've finally gotten a new phone and my camera that had been flakey for the last year decided to start working properly again - and I am happy with technology! In celebration I decided to take some pictures of my room (messy edition)

    I imported these pictures using F-Spot, which has a decent tagging interface that I should make use of to catalogue a bunch of old photos. F-Spot also happily resized and exported them to Flickr, tags and all (other stuff also supported). Hopefully this also means I'll start taking pictures again.

    read more