Other articles


  1. Money and California Propositions (2020)

    Ten years ago, I made some plots for how much money was contributed to and spent by the various proposition campaigns in California.

    I decided to update these for this election, and here's the result:

    Money Raised for CA Propositions November 2020 Election. Visualization by Paul Ivanov

    Just in case you didn't get the full picture, here is the same data plotted on a common scale:

    Money Raised for CA Propositions November 2020 Election shown with a common scale. Visualization by Paul Ivanov Money Raised for CA Propositions November 2020 Election shown with a common scale. Visualization by Paul Ivanov

    So, whereas 10 years ago, we had a total of ~$58 million on the election, the overwhelming amount of in support, this time, we had ~$662 million, an 11 fold increase!

    The Cal-Access Campaign Finance Activity: Propositions & Ballot Measures source I used last time was still there, but there are way more propositions this time (12 vs 5), and the money details are broken out by committee, with some propositions have a dozen committees. Another wrinkle is that website has protected by some fancy scraping protection. I could browse it just fine in Firefox, even with Javascript turned off, but couldn't download it using wget, curl, or python, even after setting up all of the same headers, not just the User-Agent one. I would just get something like this:

    <html>
    <head>
    <META NAME="robots" CONTENT="noindex,nofollow">
    <script
    src="/_Incapsula_Resource?SWJIYLWA=5074a744e2e3d891814e9a2dace20bd4,719d34d31c8e3a6e6fffd425f7e032f3">
    </script>
    <body>
    </body></html>
    

    or this

    <html style="height:100%"><head><META NAME="ROBOTS" CONTENT="NOINDEX,
    NOFOLLOW"><meta name="format-detection" content="telephone=no"><meta
    name="viewport" content="initial-scale=1.0"><meta http-equiv="X-UA-Compatible"
    content="IE=edge,chrome=1"></head><body style="margin:0px;height:100%"><iframe
    id="main-iframe"
    src="/_Incapsula_Resource?SWUDNSAI=9&xinfo=7-5676648-0%200NNN%20RT%281603990408902%200%29%20q%280%20-1%20-1%20-1%29%20r%280%20-1%29%20B12%284%2c316%2c0%29%20U19&incident_id=258000270014056206-28484364857902791&edet=12&cinfo=04000000&rpinfo=0&cts=P7QQVpGir0pkf9hcyxbNqBbXI4e5zF975eQFSdodvpp8NdgYKJeqrlW3TSO8Zzh5"
    frameborder=0 width="100%" height="100%" marginheight="0px"
    marginwidth="0px">Request unsuccessful. Incapsula incident ID:
    258000270014056206-28484364857902791</iframe></body></html>'
    

    Luckily, while I was working on the scraper, I took a break and found another source that already has the tabulations.

    The only catch is that this source was last updated two weeks ago, so it's not the freshest data. Also, last time I had data for both contributions and for money spent, but this summed page only has contribution totals, not spending totals (the spending figures are still there)

    But I figured it's good enough to get the big picture.

    Also, here's the python code used to generate these plots (largely reused from last time, so don't expect it to be pretty).

    # Create contributions  bar charts of committees supporting and opposing
    # various propositions on the California Ballot for November 2020
    # created by Paul Ivanov (https://pirsquared.org)
    
    # figure(0) - Contributions by Proposition (as subplots)
    # figure(2) - Contributions on a common scale
    
    import numpy as np
    from matplotlib import pyplot as plt
    import locale
    
    # fun times! without this next line, I got a
    # ValueError: Currency formatting is not possible using the 'C' locale.
    locale.setlocale(locale.LC_ALL, 'en_US.UTF-8')
    
    elec = "Nov2020"
    election =  "November 2020 Election"
    
    # This part was done by hand by collecting data from CalAccess:
    # https://www.sos.ca.gov/campaign-lobbying/cal-access-resources/measure-contributions/
    # proposition: (yes, no) 
    cont = {'14': (12810328, 250), 
            '15': (56320926, 60905901), 
            '16': (19926905, 1172614),
            '17': (1363887, 0), 
            '18': (835064, 0), 
            '19': (37794775, 45050), 
            '20': (4829677, 20471086), 
            '21': (40184953, 59379159), 
            '22': (188937777, 15896808), 
            '23': (6917438, 104405156), 
            '24': (5907002, 48368),
            '25': (13446871, 10181122)
    }
    
    def currency(x, pos):
        """The two args are the value and tick position"""
        if x==0:
            return "$0"
        if x < 1e3:
            return '$%f' % (x)
        elif x< 1e6:
            return '$%1.0fK' % (x*1e-3)
        return '$%1.0fM' % (x*1e-6)
    
    from matplotlib.ticker import FuncFormatter
    formatter = FuncFormatter(currency)
    
    yes,no = range(2)
    c = [(6.,.5,0),'blue']  # color for yes/no stance
    c = ['red','blue']  # color for yes/no stance
    c = [(1.,.5,0),'blue']  # color for yes/no stance
    a = [.9,.9]             # alpha for yes/no stance
    a = [.6,.5]             # alpha for yes/no stance
    t = ['Yes','No ']       # text  for yes/no stance
    
    raised,spent = range(2)
    title = ["Contributed to", "Spent on" ] # reuse code by injecting title specifics
    field = ['Contributions', 'Expenditures']
    
    footer ="""
    Total %s 1/1/2020-10/14/2020  (1/1/2020-10/16/2020 for Prop 15)
    Data from http://www.sos.ca.gov/campaign-lobbying/cal-access-resources/
    measure-contributions/2020-ballot-measure-contribution-totals 
    cc-by Paul Ivanov (https://pirsquared.org)
    """ # will inject field[col] in all plots
    
    color = np.array((.9,.9,.54))*.9 # spine/ticklabel color
    color = np.array((.52,.32,.12)) # spine/ticklabel color
    color = np.array((.82,.42,.12)) # spine/ticklabel color
    color = "gray"
    color = np.array((85.,107,47)) / 255 # darkolivegreen
    
    plt.rcParams['savefig.dpi'] = 200
    
    def fixup_subplot(ax,color):
        """ Tufte-fy the axis labels - use different color than data"""
        spines = list(ax.spines.values())
        # liberate the data! hide right and top spines
        [ax.spines[s].set_visible(False) for s in ["top", "right"]]
        ax.yaxis.tick_left() # don't tick on the right
    
        # there's gotta be a better way to set all of these colors, but I don't
        # know that way, I only know the hard way
        [s.set_color(color) for s in spines]
        [s.set_color(color) for s in ax.yaxis.get_ticklines()]
        [s.set_visible(False) for s in ax.xaxis.get_ticklines()]
        [(s.set_color(color),s.set_size(8)) for s in ax.xaxis.get_ticklabels()]
        [(s.set_color(color),s.set_size(8)) for s in ax.yaxis.get_ticklabels()]
        ax.yaxis.grid(which='major',linestyle='-',color=color,alpha=.3)
    
    adjust_dict = {'bottom': 0.052, 'hspace': 0.646815834767644,
     'left': 0.13732508948909858, 'right': 0.92971038073543777,
     'top': 0.94082616179001742, 'wspace': 0.084337349397590383}
    
    # subplots for each proposition (Fig 0 and Fig 1)
    col = 0 
    f = plt.figure(col); f.clf(); f.dpi=100;
    for i,p in enumerate(cont):
        ax = plt.subplot(len(cont),1, i+1)
        ax.clear()
        #p = i+14    #prop number
        for stance in [yes,no]:
            plt.bar(stance, cont[p][stance], color=c[stance], linewidth=0,
                    align='center', width=.1, alpha=a[stance])
            lbl = locale.currency(round(cont[p][stance]), symbol=True, grouping=True)
            lbl = lbl[:-3] # drop the cents, since we've rounded
            ax.text(stance, cont[p][stance], lbl , ha='center', size=8)
    
        ax.set_xlim(-.3,1.3)
        ax.xaxis.set_ticks([0,1])
        ax.xaxis.set_ticklabels(["Yes on %s"%p, "No on %s"%p])
    
        # put a big (but faded) "Proposition X" in the center of this subplot
        common=dict(alpha=.1, color='k', ha='center', va='center', transform = ax.transAxes)
        ax.text(0.5, .9,"Proposition", size=8, weight=600, **common)
        ax.text(0.5, .50,"%s"%p, size=50, weight=300, **common)
    
        ax.yaxis.set_major_formatter(formatter) # plugin our currency labeler
        ax.yaxis.get_major_locator()._nbins=5 # put fewer tickmarks/labels
    
        fixup_subplot(ax,color)
    
    adjust_dict.update(left=0.13732508948909858,right=0.92971038073543777)
    f.subplots_adjust( **adjust_dict)
    
    # Figure title, subtitle
    extra_args = dict(family='serif', ha='center', va='top', transform=f.transFigure)
    f.text(.5,.99,"Money %s CA Propositions"%title[col], size=12, **extra_args)
    f.text(.5,.975,election, size=9, **extra_args)
    
    #footer
    extra_args.update(va='bottom', size=6,ma='center')
    f.text(.5,0.0,footer%field[col], **extra_args)
    
    f.set_figheight(20.); f.set_figwidth(5); f.canvas.draw()
    f.savefig('CA-Props-%s-%s-Subplots.png'%(elec, field[col]))
    
    # all props on one figure (Fig 2 and Fig 3)
    f = plt.figure(col+2); f.clf()
    adjust_dict.update(left= 0.045,right=.98, top=.91, bottom=.12)
    f.subplots_adjust( **adjust_dict)
    f.set_figheight(6.)
    f.set_figwidth(16)
    
    extra_args = dict(family='serif', ha='center', va='top', transform=f.transFigure)
    f.text(.5,.99,"Money %s CA Propositions"%title[col], size=12, **extra_args)
    f.text(.5,.96, election , size=9, **extra_args)
    
    footer = footer.replace("/\n", "/") #.replace("\nc", "c")
    #footer = footer.replace("\n", "") + "\n"
    extra_args.update(ha='center', va='bottom', size=6,ma='center')
    #f.text(adjust_dict['left'],0.0,footer%field[col], **extra_args)
    f.text(.5,0.0,footer%field[col], **extra_args)
    
    ax = plt.subplot(111)
    for stance in [yes,no]:
        abscissa=np.arange(0+stance*.30,len(cont),1)
        total = sum([x[stance] for x in cont.values()])
        lbl = locale.currency(round(total),True,True)
        lbl = lbl[:-3] # drop the cents, since we've rounded
        lbl = t[stance]+" Total "+ lbl.rjust(12)
        plt.bar(abscissa,[cont[p][stance] for p in cont], width=.1, color=c[stance],
                alpha=a[stance],align='center',linewidth=0, label=lbl)
        for i,p in enumerate(cont):
            lbl = locale.currency(round(cont[p][stance]), symbol=True, grouping=True)
            lbl = lbl[:-3] # drop the cents, since we've rounded
            #ha = 'center' if i != 2 else "right" # tweek by hand to make numbers show up
            ax.text(abscissa[i], cont[p][stance], lbl , ha="center",
                    size=8,rotation=00)
    
    ax.set_xlim(xmin=-.3)
    ax.xaxis.set_ticks(np.arange(.15,len(cont),1))
    ax.xaxis.set_ticklabels(["Prop %s"%p for p in cont])
    fixup_subplot(ax,color)
    
    # plt.legend(prop=dict(family='monospace',size=9)) # this makes legend tied
    # to the subplot, tie it to the figure, instead
    handles, labels = ax.get_legend_handles_labels()
    l = plt.figlegend(handles, labels,loc='upper right',prop=dict(family='monospace',size=9))
    l.get_frame().set_visible(False)
    ax.yaxis.set_major_formatter(formatter) # plugin our currency labeler
    f.canvas.draw()
    f.savefig('CA-Props-%s-%s.png'%(elec, field[col]))
    
    plt.show()
    
    permalink
  2. pheriday 3: infrastructure

    Looks like we can't inline audio for your browser. That's cool, just find the direct file links below.

    paul's habitual errant ramblings (on Fr)idays

    pheridays: 3

    2020-04-10: A week ago, I recorded a 5 minute audio segment of some stuff I've been thinking about, but when I started to write it up I stumbled into and kept dropping down a deep technostalgic hole.

    fall down along with me:

    https://pirsquared.org/blog/pheriday-infrastructure.html

    The recording is just shy of five minutes long, you can also download it in different formats, depending on your needs, if the audio tag above doesn't suit you:

    https://pirsquared.org/pheridays/2020-04-03.ogg (2.9 Mb)
    https://pirsquared.org/pheridays/2020-04-03.mp3 (4.5 Mb)
    https://pirsquared.org/pheridays/2020-04-03.m4a (6.3 Mb)

    --

    Stuff I mentioned in the audio:

    Propellor - "configuration management system using Haskell and Git" by Joey Hess

    OpenWRT - specifically - reducing Bufferbloat

    Mumble - "a free, open source, low latency, high quality voice chat application."

    sourcehut.org - "the hacker's forge" also know as sr.ht by Drew DeVault

    Jitsi - "Multi-platform open-source video conferencing"

    OpenFire - "real time collaboration (RTC) server licensed under the Open Source Apache License." Extensible XMPP server, with plugins, like a Jitsi-based video meeeting one claled OpenFire Meetings.


    Though this is the fourth installment, the last time I recorded and posted a rambling was back almost 8 years ago! In fact, it was 2012-08-03, so 7 years and 8 months, to the day.


    Having control of your infrastructure is a longtime thread for me.

    For starters - there's the bicycle. That's been my primary and preferred mode of transportation for 30 years. As a kid, I was empowered by the sense of freedom, independence, and self-sufficiency that came with a bike. All these years later, I'm still a fan. You can see just how happy I am on a bike at the top of this interview , thanks to a sweet photo that was taken by Robert Sexton right by the Golden Gate Bridge at the end of the Lucas Valley Populaire in 2015.

    Those of you who knew me back in college might remember how at UC Davis I ran my own "pirate" internet radio station - KPVL - with the cheeky tagline of "More broadcasters than listeners". (I say "pirate" because it has not relation to the actual KPVL radio station). But there are earlier remnants and traces of my efforts to exercise control and build my own reality.

    I think it was in 1999 that my brother Mike and I started using Redhat (6), then Mandrake Linux 6.5, dual booting on a computer at home and I separately around the same time I got myself an sdf.org account. Though I wasn't sophisticated enough to have a constant internet connection in high school, I was lucky enough to get an account on Robert Chin's laya.com server. The url was - p.laya.com - it's long gone, but luckily, Archive.org has a copy from 2001.

    Wow. I just took a look and so much came flooding back.

    Here's the thing: April is an anniversary of sorts for me. Back in 1999, it marks my first time breaking anonymity and pseudonimity and using my real name on the internet. I've written about this before under the title of Publisher's block ten years ago - just about half way between now and then. This time, though, let me inline the piece I linked to as proof of the deliberate nature of my lack of anonymity.

    An account of my life at 15, as I live it.
    traces of my awareness of the world, I can look back at later
    My first attempt at a memoir
    My goal is to capture my many thoughts emotions, behaviors, incidents, and
    acquaintances
    and to arrive only at an exponential number of those,
    hoping yet being afraid that it might be zero
    making everything about me: one
    

    I'm glad I can now reflect on the kind of kid I was, thanks to the amazing folks who had the foresight to start archiving all of the web for The Way Back Machine. I used that p.laya.com page as a todo list and notes for myself using a hipster combination of the default file index listing with a FOOTER.html. It was captured in 2001, I was in 17, but some of this was written when I was 15 or 16 (I found contents from November 2000), I make mention to my then freeshell.org account (it's now been ivanov@ since 2012). I link to the source code of a MUD - ftp://ftp.game.org/pub/mud/diku/merc/rom/tartarus/tartarus.tgz - which is a broken link now, but I found a mirror over here: which is amazing, because just a week or two ago, I was hanging out with fellow SciPy 2020 program co-chairs Madicken Munk and Gil Forsyth over video chat after one of our meetings and I was happily reporting about how one of the positive things to come out of the shelter in place for me is that "I've fixed my mutt configuration and started using it again!" - but they both heard "mutt" as "MUD" and got very excited by that prospect. So much so that we all agreed that we'll have to follow up and actually follow through to build a MUD. And I brought up how at some point in high school I was mildly active in a pair of MUDs, and wanted to make my own, but never got around to it.

    The last link I left for myself on there points to pinkmonkey.com - a homeschooling resource - which is probably handy for the parents with little ones these days.

    Here's the most concrete infrastructure project I can find from then: I collected bookmarks from my friends to share them. The "service" lived at http://p.laya.com/bookmarks - and predates del.icio.us and pinboard. I bet I "advertised" it in my AIM profile.

    If you're curious, there's a link to the archive.org copy near the end of this post, but I had this urge to show it to you much closer to its original glory.

    Let me set the scene: It's Friday in April, the year is 2020, I'm running Windows 10 on my work laptop in poorly connected home in California, where a pandemic has most of the state's residents staying put at home for the several weeks already, and I decided to make a screenshot using the tool du jour of yesteryear

    Netscape Navigator!

    The timestamp on my bookmark website says I last updated it on: Thu Aug 30 20:14:14 PDT 2001

    OldVersion.com tells me that the latest release for Windows that Netscape 4.79 was released in November of that year, and the closest antecedent version available is 4.72 (from February 2000).

    I downloaded it and tried fiddling around with the compatibility settings, but without any luck.

    Then I tried 4.79, and nope, that didn't work, either. So then I tried Netscape 6.01 - release February 2001.

    I happened to have Chrome running at the time because in Firefox I have 1500 tabs open -- fifteen hundred and seven! ;) -- whereas in Chrome it's under 500, so I was trying to tread lightly. How do I know these numbers? For Chrome I found an extension that allows me to copy into the clipboard all open tabs' urls as plain text. It helpfully announces how many such tabs were copied. In Firefox one of the webextension examples gives you a counter.

    Do you remember the web without tabs? Time was, you wanted to visit another webpage, you got two option: you navigate away from whatever you're looking at now, or you hit Ctrl-N to make an new window. I think most people used one or a few windows. But you were not gonna be crazy and open more than a dozen windows. I would have, and probably tried but I couldn't. And session saving across crashes or clean exits? Forget it! That what your history and bookmarks are for, grasshopper.

    But let's get back to the task at hand: this was the lower right of my screen...

    Downloaded Netscape601.exe

    and I decide to start taking screenshots of this journey, click it, and let Windows 10 apply the compatibility settings, and then I'm faced with

    the most improbable error message:

    Setup detected another instance of Netscape 6 is currently running

    :)

    WAT?!

    I didn't think Chrome had any ancestry shared with the Mosaic super-tree, but whatever - you can't exactly argue with software from 2001, and I have an important screenshot to take...

    So now I've quit Chrome, just in case, and going to retry....

    Setup detected another instance of Netscape 6 is currently running

    no dice....

    Damn, what could it be...I've got the Bloomberg Terminal open, I know portions of it are built on Chromium browser technology (had to look it up if this was officially stated somewhere - it is). Ok, so maybe that's what causing the false positive? I close that, and...

    Setup detected another instance of Netscape 6 is currently running

    ...

    I hardly have anything open anymore ...is it VLC?

    ...

    Setup detected another instance of Netscape 6 is currently running

    nope... Ok, what's left still open... Snipping tool I'm using to capture this epic adventure, a few WSL Debian console windows... the voice recorder that started this post... Task manager and Sysinternals' Process Explorer - where I was checking if perhaps somehow the failed attempt at running what was probably a 16 bit version of Navigator 4.72 was still lingering somewhere... SumatraPDF, Windows Terminal (Preview), gVim, and ...

    Zotero?!?

    BINGO!!!!!!!!!

    Initial Netscape 6 Setup screen

    Initial Netscape 6 Setup screen

    Oh right - I guess Zotero uses XUL technology. I didn't really think much about it, but Zotero did start off life as a Firefox extension, and the standalone version came out later, makes sense that it would have grabbed a browser when it struck out on its own.

    At this point I had already sent Madicken, who works at NCSA where Mosaic, the progenitor of Netscape hails from, the first two images... So I wanted to play with fire a bit....

    Now that I've closed Zotero - can I have Firefox 74 open while installing Netscape 6.01?

    Setup detected another instance of Netscape 6 is currently running

    Rats! same error...

    how about Chrome again?

    Initial Netscape 6 Setup screen

    Initial Netscape 6 Setup screen

    oh yeah! Sweet. The world makes sense again.

    Back to the setup.exe...

    I scroll through the EULA - and randomly stop on this section:

    12. HIGH RISK ACTIVITIES.  The Product is not
    fault-tolerant and is not designed, manufactured or
    intended for use or resale as on-line control
    equipment in hazardous environments requiring
    fail-safe performance, such as in the operation of
    nuclear facilities, aircraft navigation or
    communication systems, air traffic control, direct
    life support machines, or weapons systems, in which
    the failure of the Product could lead directly to
    death, personal injury, or severe physical or
    environmental damage ("High Risk Activities").
    

    By the way - here's a good idea I came across a few months ago: throw EULAs (End User License Agreements) into some publicly indexed version control repo (I saw folks using gists for just that sort of thing: here's the Netscape 6.01 EULA.txt)

    Let the folks at Redmond host it.

    Fine. I click next...

    Interesting - there's a "Read Me" button... I do as I'm told, so I click it.

    Oh, a README.txt popped up in Notepad.exe - full contents of that are in that same gist as the EULA

    but in them, there's a link to the full release notes over at

    http://home.netscape.com/eng/mozilla/ns6/relnotes/6.0.html

    Let's try to go there now:

    "Yahoo! will be right back... Thank yo for your patience."

    Wat?!

    Yeah, I'm sure Yahoo! engineers are jumping right on that.

    How did a Netscape site ended up redirecting to Yahoo...

    oh right, so AOL bought Netscape (1999), merged with Time Warner in 2001, was spun out again in 2009, after some rough times, and then purchased by Verizon in 2015. In the meantime, Yahoo acquired Geocities (1999) and shut it down in 2009 (yes, I'm still mad! All I remember was that it had some crispy banners, one of which was a scan of a sweet pencil lettering I made of my nick at the time - "ShadowKnight"). No one really cares what happened to Yahoo in the interim, aside from some massive data breaches, until finally, Verizon bought Yahoo in 2017 and merged AOL and Yahoo into one division.

    Ok, so fine, maybe fine folks at The Internet Archive (archive.org) can help us with the WayBackMachine.

    And indeed we can see what it looked like originally. This site has seen so many redirects over the years - it'd be a fun exercise to go through all of the indexed versions of this kind of site to see how people tried to preserve links. For example, I found out in 2010 it 301s ("Temporary redirect") to http://www.netscape.com/eng/mozilla/ns6/relnotes/6.0.html which then 302s ("Permanent redirect") to http://www.propeller.com/eng/mozilla/ns6/relnotes/6.0.html - which was indexed but happens to be a 404 ("Page not found") error page, at least in 2008. But this wasn't what we came here for, so this yak can roam free among the hills, the valleys, and the caverns of our minds.

    ... for now...

    Where were we?

    Oh right, we have to choose an install option. Back in the day I might have clicked "recommended" here, but we're not back in the day, and who wants to play life on easy mode? Let's go custom to see what the options are (I am a control freak, after all)

    Custom Install Option selected

    and immediately get another pop up:

    "Create install directory?" prompt

    Whoa, let's pause here for a moment.

    I am digging such a consent model. The installer is establishing trust: it will not try to do anything behind my back and without my permission. I'm sure it will never abuse that trust. As a 2001 user, I sure am glad that folks involved with computing have such a well-develop sense of ethics. In 2001, the Future is bright. Computing will be filled with transparency. Consumer software and services will be built by folks with a strong moral compass. These are people with principles. With the dot-com bubble burst, we've swiftly inoculated tech from sleazy opportunists. It won't fall victim to the excesses and greed rivaling Wall Street in the 80s...

    Which reminds me - how is it that this brilliant video only has 277 thousand views? Here's a excerpt:

    Your users won't always understand just how much economic sense it makes to sell them out. And you don't want to alienate them, that would drive down their value.

    By the way - the embed code I used above uses the youtube-nocookie.com domain - which is still from the folks at Goolag, but does what it says on the tin and doesn't issue cookies. Also, did you know there used to be a way to disable those annoying related video links from popping up at the end of the video? It's true. You used to be able to just append a rel=0 query parameter to not show related videos. But the corporate overlord bean counters didn't like that. They had to make sure that kids would get glued to the site by feeding them progressively conspiratorial garbage content. So that watching any video would nearly guarantee to pull them into the black hole cesspool of maximally "engaging" "content". What was that quote about users again? Ah yes:

    Your users won't always understand just how much economic sense it makes to sell them out. And you don't want to alienate them, that would drive down their value.

    Ok, so, in fairness, rel=0 query parameter still does something. It limits the suggested videos to the channel they are from. That's good. But what happens if we follow one of those links? First, we end up on the full youtube site, so that means the cookies are back. Hurray for surveillance capitalism! Also, the recommendations on the right are curated specifically for us, and not limited to the channel the previous video was one. Goodbye, rel=0!

    So I have to install Navigator, but I can unselect Mail, Instant Messenger, and Spell Checker... I don't need mail, but whatever, let's just go with the defaults.

    Oh, look at all this wonderful bundled crapware. Just in case you had any doubts.

    That last one made me throw up a little in my mouth.

    I opt for just the classic skin - and it tells me that the total download size will be 9959 K.

    I thought about censoring this next screenshot, but 15 year old me wouldn't have like that... What's in the shot is in the shot..

    Alright, and when the installation finished here's what we're greeted with:

    Did software in 2001 try to phone home? "activation.netscape.com could not be found." sure seems so.

    Yes, No, Cancel?

    What if I cancel?

    Alright, let's go for broke and get that retro look...

    Remember all those redirects? Well the browser froze when I got overzealous, clicked on "Interact" at the bottom there and chose to open chat... And on the next load, it crashed... And again...

    Netscape 6 Frozen on Yahool Will be be right back page

    Netscape 6 Frozen on Yahool Will be be right back page It just kept crashing...

    In case anyone else gets stuck on the same issue ;) I got around this by using the Profile Manager, where I had the option to start the browser in Work Offline mode. Then I turn the "Work Online" option on after the browser loaded (which you can do by plugging together that cute outlet pair on the bottom left).

    I do some ego surfing and go to my own site first.

    http://pirsquared.org

    pirsquared.org using Netscape 6

    pirsquared.org using Netscape 6

    I got too fancy with my unicode... But hey, this is totally functional.

    I made this Loading gif via a screen capture tool and then it finally clicked that not only did I not use Netscape 6 - I remember most everyone's experience was to stick to the 4.x series, because it was so much more usable and not bloated with nonsense, etc, etc.

    Alright, but at least I got 6 to run on Windows 10 and that works...When it doesn't crash, anyway... But I did get a error about youtube-nocookie.com... (some of the time, at least)...

    And then I realized that I can't go to any site that has https... Because... you know, the protocol that provides that 's' has changed over the years, and our 2001 browser could do SSL 2 or 3 or TLS 1.0... But my website uses TLS 1.3...

    I couldn't run to duckduckgo, either, since it redirects plain http to the https endpoint and that also runs TLS 1.3... I couldn't even go to archive.org to view my old site directly on the way back machine, because archive.org run TLS 1.2 at the moment.

    It's difficult to find any place that still runs such outdated standards... I tried to search for just a TLS 1.0 test server - but didn't find anything suitable... But then I happen to flip through the recent changelog for Firefox:

    Firefox 74 Reverting deprecation of TLS 1.0 and 1.1

    Cool - so now we know if we want to find TLS 1.0 and 1.1 website, we should turn to the government of... damn a specific country wasn't specified...

    But wait a minute... Firefox 74.0 came out on March

    How did Mozilla release an update to a version of Firefox that was in the hands of a bunch of users without... umn...what's the word I'm looking for here...you know, that thing no one seems to think is a thing anymore... user consent?

    How Mozilla released an update without user consent

    https://wiki.mozilla.org/Firefox/Normandy/PreferenceRollout

    This is the way consent ends
    This is the way consent ends
    This is the way consent ends
    Not with a bang but a whimper
    

    I'm late to the party - this has been going on for about three years - with what I now recall was caused a bit of a splash back in 2017 (Drew DeVault covers in "Firefox is on a slippery slope"). But I didn't know the extent of it. Who has the time to pay attention to the way in which all the software they use changes in anti-social ways.

    Anyway, if you don't want the fine folks at Firefox to change your preferences out from under you, I think you go to about:config and switch the app.normandy.enabled setting to false. And if you're interested in specifics of how you've been a guinea pig: about:studies will tell you. And you can go to about:preferences#privacy to disable them.

    But I digress...

    Let's wrap this up...

    At this point, I used my 2020 browser to grab historical snapshot of my old bookmarks site, stripping off the tastefully annotated Way Back Machine user interface insertions, and serve it locally over http via python -m http.server, making sure to change the URL bar to make a historically accurate re-enactment. And now that you know how I got here, you can fully appreciate the effort that went into this next screenshot:

    My bookmarks Gallore site in Netscape 6

    My bookmarks Gallore site in Netscape 6

    So much so, that I couldn't resist making a video a scroll through :

    You can download it as webm, mp4, ogg, or gif.

    If you're interested - you can go http://p.laya.com/bookmarks/ (archival copy) or see the rest of p.laya.com from 2001

    That's all


    The ongoing crisis has been a circuit breaker to our usual patterns. I am taking advantage of this affordance to experiment with and establish channels of communications that are not controlled by others.


    permalink
  3. PyCon2019 poem

    I'm back in Cleveland for another Pycon. Yesterday was my first full day here. Along with Matt Seale, I was a helper at Matthias Bussonnier tutorial ("IPython and Jupyter in Depth: High productivity, interactive Python). The sticky system is efficient at signaling when someone in a classroom needs help, and a lot of folks don't know that this practice was popularized by Software Carpentry workshops and continues to be used at The Carpentries.

    Explanation of The Carpentries Sticky system: "It is easy to get help.
We use a system of sticky notes to allow learners to seek help without having to
call attention to themselves or interrupt the Instructor’s flow. Learners are
issued sticky notes of two different colours - generally red and green - and use
them to signal ‘it’s all good’ or ‘I need help’. When a learner posts a red
sticky, a helper in the room will assist them to get back on track"

    I stepped out for a coffee refill and bumped into a large contingent of Bloomberg folks I'd never met (Princeton office). I guess we have something like 90 people at the conference this year, and I made the usual and true remark about how I go to conferences to meet the other people who work at our company. Then after his tutorial concluded, Matthias and I bumped into Tracy Teal, exchanged some stickers, and chatted about The Carpentries, Jupyter, organizing conferences, governance and sponsorship models, and a bunch of other stuff.

    Matthias was a good influence in trying to participate in the swag bag stuffing, but it turned out we had almost an hour to kill maim (we decided injury's a better fate than death for that hour). Luckily, we bumped into John Lam and proceeded to chat away and entertain one another. The swag bag stuffing was really fun - highly recommend it to everyone! Smart people performing a task that allows for lots of banter and brief social interaction turn while there's music and a chance to move around turns out to be a good way to have time continue flying by. Too many people to name here, but Scopz was there and we made terrible puns, as usual.

    After the swag bag stuffing party, reconnected with Tracy and we chatted some more while walking around and picking up swag (mostly socks), joined by Gil Forsyth who handed me Xonsh sticker, bumped into Saul Shanabrook. Then stopped by to say hi to Jess, a friend of Anna Vu's, whom I only ever see at PyCon), and then chatted with Scott Sanderson, met Hugo Bowne-Anderson, and got to briefly greet Jonathan Helmus.

    I walked Matthias to his hotel, then went back to Butcher and the Brewer (again, Matthias joined me for dinner there the night before), and wrote out the poem you'll find below. After I wrote it, a bunch of my Bloomberg colleagues showed up, and I ended up chatting for long time with John Purviance. In keeping with tradition, though we work at the same company, I met John by accident at a coffee shop during last year's Pycon (he works in the New York office, whereas I'm in SF).

    One day down, and it's already been a wonderful Pycon.

    Pycon 2019

    May 2nd, Pycon Cleveland Two, as well
    The social wealth observed, created
    not jaded, lots to say and listen to
    Bring energy and silliness, too
    My objective to subjectively inject 
    a crack and tear decorum fabric
    be vulnerable
    a buzzing bee
    break up the swag bag stuffing monotony
    walk the path, switch sides, talk trash
    Great folks, give stickers, smiles, rehash
    A splash of warmth, a squak, a "BAM!"
    A nudge - we jam enjoying this team work
    not work at all - a play - a sandbox
    roles all clear yet much room left 
    to improvise
    Devise a way to make us laugh
    each finds their path
    rocking forth and back
    swaying fro and too
    a rhythm of a train a-moving
    Clapping celebration when we're done
    "Hey-hey! We're done!" - a dinner earned
    and well deserved - many socks collected
    New people met and old friends reconnected
    Tutorials and opening reception done
    and now Pycon 2019's officially begun
    
    permalink
  4. Get in it

    Two weeks ago, Project Jupyter had our only planned team meeting for 2018. There was too much stuff going on for me to write a poem during the event as I had in previous years (2016, and 2017), so I ended up reading one of the pieces I wrote during my evening introvert breaks in Cleveland at PyCon a few weeks earlier.

    Once again, Fernando and Matthias had their gadgets ready to record (thank you both!). The video below was taken by Fernando.

    click to view the video

    Get in it

    Time suspended
    Gellatinous reality - the haze
    submerged in murky drops summed
    in swamp pond of life
    
    believe and strive, expand the mind
    A state sublime, when in your prime you came to
    me and we were free to flow and fling our
    cares, our dreams, our in-betweens, our
    rêves perdues, our residue -- the lime of light
    the black of sight -- all these converge and
    merge the forks of friction filled with fright
    and more -- the float of logs that plunges deep
    beyond the fray, beyond the keep -- a leap of faith
    the lore of rite, with passage clear, let
    fear subside, the wealth of confidence will
    rise and iron out wrinkles of doubt
    
    Commit to change and stash your pride
    then push your luck, and make amends.
    Branch out your thoughts, reset assumptions
    then checkout.
    
    The force of pulls t'wards master class
    Remote of possibilities. Rehash the past
    Patch up the present -- what's the diff?
    
    There's nothing left -- except to glide -- and
    soar beyond your frame of mind.  try not to pry
    cry, freedom, cry.
    
    permalink
  5. SciPy 2018 dates and call for abstracts

    I'm helping with next year's SciPy conference, so here are the details:

    July 9-15, 2018 | Austin, Texas

    Tutorials: July 9-10, 2018
    Conference (Talks and Posters): July 11-13, 2018
    Sprints: July 14-15, 2018
    

    SciPy 2018, the 17th annual Scientific Computing with Python conference, will be held July 9-15, 2018 in Austin, Texas. The annual SciPy Conference brings together over 700 participants from industry, academia, and government to showcase their latest projects, learn from skilled users and developers, and collaborate on code development. The call for abstracts for SciPy 2018 for talks, posters and tutorials is now open. The deadline for submissions is February 9, 2018.

    Talks and Posters (July 11-13, 2018)

    In addition to the general track, this year will have specialized tracks focused on:

    • Data Visualization
    • Reproducibilty and Software Sustainability

    Mini Symposia

    • Astronomy
    • Biology and Bioinformatics
    • Data Science
    • Earth, Ocean and Geo Science
    • Image Processing
    • Language Interoperability
    • Library Science and Digital Humanities
    • Machine Learning
    • Materials Science
    • Political and Social Sciences

    There will also be a SciPy Tools Plenary Session each day with 2 to 5 minute updates on tools and libraries.

    Tutorials (July 9-10, 2018)

    Tutorials should be focused on covering a well-defined topic in a hands-on manner. We are looking for awesome techniques or packages, helping new or advanced Python programmers develop better or faster scientific applications. We encourage submissions to be designed to allow at least 50% of the time for hands-on exercises even if this means the subject matter needs to be limited. Tutorials will be 4 hours in duration. In your tutorial application, you can indicate what prerequisite skills and knowledge will be needed for your tutorial, and the approximate expected level of knowledge of your students (i.e., beginner, intermediate, advanced). Instructors of accepted tutorials will receive a stipend.

    Mark Your Calendar for SciPy 2018!

    permalink
  6. SciPy 2017 tips

    After missing it for a couple of years, I am happy to be back in Austin, TX for SciPy this week!

    Always invigorating and exhilarating, Scientific Computing with Python (SciPy) has remained a top quality venue for getting together with fellow Pythonistas, especially the academically-bent variety.

    As a graduate student eight years ago, I was fortunate enough to be one of receive sponsorship and attended my first SciPy - SciPy 2009. This was the last time it was held at CalTech in Pasadena, CA.

    @ivanov/status/2648139681

    The following year, in 2010, at the first SciPy held in its now usual spot in Austin, TX, each attendee got a bottle of delicious salsa!

    SciPy2010 Salsa Stack

    Here are some oy my thoughts about attending this wonderful conference.

    Conference Tips

    bring a sweatshirt -- Yes, I know Austin's hot, but at the AT&T center, they don't mess around and crank the air conditioning all the way up to 11!

    join the slack group -- This year, there's a Slack group for SciPy: the link to join is in a pair of emails with the titles "Getting the most out of SciPy2017" and "Getting the most out of SciPy2017-UPDATED", both from SciPy2017 Organizers. So far at the tutorials slack has served as a useful back channel for communicating repo URLs and specific commands to run, signaling questions without interrupting the speakers' flow.

    engage with others during the breaks, lunch, etc -- There are lots of tool authors here and we love chatting with users (and helping you become contributors and authors yourselves). Not your first SciPy and feeling "in-your-element"? Make the effort to invite others into the conversations and lunch outings you're having with old friends - we're all here because we care about this stuff.

    take introvert breaks (and be considerate of others who may be doing the same) - I'm an introvert. Though I enjoy interacting with others (one-on-one or in small groups is best for me), it takes a lot of energy and at some point, I run out of steam. That's when I go for a walk, stepping away from the commotion to just have some quiet time.

    be kind to yourself (especially at the sprints) -- Between the tutorials, all of the talks, and the sprints that follow, there will be a flurry of activity. Conferences are already draining enough without trying to get any work done, just meeting a bunch of new people and taking in a lot of information. It took a lot of false starts for me to have productive work output at sprints, but the best thing I've learned about them is to just let go of trying to get a lot done. Instead, try to get something small and well defined done or just help others.

    Stuff to do in Austin

    The conference already has a great list of Things to do in Austin, as well as Restaurants, so I'll just mention a few of my personal favorites.

    Barton Springs Pool. Take a nice dip in the cool waters, and grab a delicious bite from one of the food trucks at The Picnic food truck park.

    Go see the bats. The Congress Ave bridge in Austin is home to the largest urban bat colony in the world. You can read more about this here, but the short version is that around sunset (8-9pm) - a large number of bats stream out from underneath the bridge to go feed on insects. Some days, they leave in waves (this Saturday there were two waves, the first was smaller, but many people left thinking that was the entire show).

    I hope you enjoy SciPy2017!

    permalink
  7. June 1st, 2017

    We had another biannual Jupyter team meeting this week, this time it was right nearby in Berkeley. Since I had read a poem at the last meeting, I was encouraged to keep that going and decided to make this a tradition. Here's the result, as delivered this past Friday, recorded by Fernando Pérez (thanks, Fernando!).

    June 1st, 2017

    We struggle -- with ourselves and with each other
    we plan -- we code and write
    the pieces and ideas t'wards what we think is right
    but we may disagree -- about
    the means, about the goals, about the
    shoulders we should stand on --
    where we should stand, what we should stretch toward
    shrink from, avoid, embrace --
    a sense of urgency - but this is not a race
    
    We can't erase the past
    but we have built this future present
    There's much to learn, to do...
    
    Ours not the only path, no one coerced you here
    You chose this -- so did I and here we are --
    still at the barricades and gaining ground
    against the old closed world:
    compute communication comes unshackled
    
    permalink
  8. November 9th, 2016

    Two weeks ago, I went down to San Luis Obispo, California for a five day Jupyter team meeting with about twenty five others. This was the first such meeting since my return after being away for two years, and I enjoyed meeting some of the "newer" faces, as well as catching up with old friends.

    It was both a productive and an emotionally challenging week, as the project proceeds along at breakneck pace on some fronts yet continues to face growing pains which come from having to scale in the human dimension.

    On Wednesday, November 9th, 2016, we spent a good chunk of the day at a nearby beach: chatting, decompressing, and luckily I brought my journal with me and was able to capture the poem you will find below. I intended to read it at a local open mic the same evening, but by the time I got there with a handful of fellow Jovyans for support, all of the slots were taken. On Friday, the last day of our meeting, I got the opportunity to read it to most of the larger group. Here's a recording of that reading, courtesy of Matthias Bussonnier (thanks, Matthias!).

    November 9th, 2016

    The lovely thing about the ocean is
    that it
    is
    tireless 
    It never stops
    incessant pendulum of salty foamy slush
    Periodic and chaotic
    raw, serene 
    Marine grandmother clock  
    crashing against both pier
    and rock
    
    Statuesque encampment of abandonment
    recoiling with force
    and blasting forth again
    No end in sight
    a train forever riding forth
    and back
    along a line
    refined yet undefined
    the spirit with
    which it keeps time 
    in timeless unity of moon's alignment
    
    I. walk. forth.
    
    Forth forward by the force
    of obsolete contrition
    the vision of a life forgotten
    Excuses not
    made real with sand, wet and compressed
    beneath my heel and toes, yet reeling from
    the blinding glimmer of our Sol
    reflected by the glaze of distant hazy surf
    upon whose shoulders foam amoebas roam
    
    It's gone.
    Tone deaf and muted by
    
    anticipation
    each coming wave
    breaks up the pregnant pause
    And here I am, barefoot in slacks and tie
    experiencing sensations
    of loss, rebirth and seldom 
    kelp bulbs popping in my soul.
    
    permalink
  9. Jupyter's Gravity

    I'm switching jobs.

    For the past two years I've been working with the great team at Disqus as a member of the backend and data teams. Before that, I spent a half-dozen years mostly not working on my thesis at UC Berkeley but instead contributing to to the scientific Python ecosystem, especially matplotlib, IPython, and the IPython notebook, which is now called Jupyter. So when Bloomberg reached out to me with a compelling position to work on those open-source projects again from their SF office, such a tremendous opportunity was hard to pass up. You could say Jupyter has a large gravitational pull that's hard to escape, but you'd be huge nerd. ;)

    I have a lot to catch up on, but I'm really excited and looking forward to contributing on these fronts again!

    permalink

Page 1 / 3 »