Other articles


  1. SciPy 2018 dates and call for abstracts

    I'm helping with next year's SciPy conference, so here are the details:

    July 9-15, 2018 | Austin, Texas

    Tutorials: July 9-10, 2018
    Conference (Talks and Posters): July 11-13, 2018
    Sprints: July 14-15, 2018
    

    SciPy 2018, the 17th annual Scientific Computing with Python conference, will be held July 9-15, 2018 in Austin, Texas. The annual SciPy Conference brings together over 700 participants from industry, academia, and government to showcase their latest projects, learn from skilled users and developers, and collaborate on code development. The call for abstracts for SciPy 2018 for talks, posters and tutorials is now open. The deadline for submissions is February 9, 2018.

    Talks and Posters (July 11-13, 2018)

    In addition to the general track, this year will have specialized tracks focused on:

    • Data Visualization
    • Reproducibilty and Software Sustainability

    Mini Symposia

    • Astronomy
    • Biology and Bioinformatics
    • Data Science
    • Earth, Ocean and Geo Science
    • Image Processing
    • Language Interoperability
    • Library Science and Digital Humanities
    • Machine Learning
    • Materials Science
    • Political and Social Sciences

    There will also be a SciPy Tools Plenary Session each day with 2 to 5 minute updates on tools and libraries.

    Tutorials (July 9-10, 2018)

    Tutorials should be focused on covering a well-defined topic in a hands-on manner. We are looking for awesome techniques or packages, helping new or advanced Python programmers develop better or faster scientific applications. We encourage submissions to be designed to allow at least 50% of the time for hands-on exercises even if this means the subject matter needs to be limited. Tutorials will be 4 hours in duration. In your tutorial application, you can indicate what prerequisite skills and knowledge will be needed for your tutorial, and the approximate expected level of knowledge of your students (i.e., beginner, intermediate, advanced). Instructors of accepted tutorials will receive a stipend.

    Mark Your Calendar for SciPy 2018!

    read more
  2. SciPy 2017 tips

    After missing it for a couple of years, I am happy to be back in Austin, TX for SciPy this week!

    Always invigorating and exhilarating, Scientific Computing with Python (SciPy) has remained a top quality venue for getting together with fellow Pythonistas, especially the academically-bent variety.

    As a graduate student eight years ago, I was fortunate enough to be one of receive sponsorship and attended my first SciPy - SciPy 2009. This was the last time it was held at CalTech in Pasadena, CA.

    The following year, in 2010, at the first SciPy held in its now usual spot in Austin, TX, each attendee got a bottle of delicious salsa!

    SciPy2010 Salsa Stack

    Here are some oy my thoughts about attending this wonderful conference.

    Conference Tips

    bring a sweatshirt -- Yes, I know Austin's hot, but at the AT&T center, they don't mess around and crank the air conditioning all the way up to 11!

    join the slack group -- This year, there's a Slack group for SciPy: the link to join is in a pair of emails with the titles "Getting the most out of SciPy2017" and "Getting the most out of SciPy2017-UPDATED", both from SciPy2017 Organizers. So far at the tutorials slack has served as a useful back channel for communicating repo URLs and specific commands to run, signaling questions without interrupting the speakers' flow.

    engage with others during the breaks, lunch, etc -- There are lots of tool authors here and we love chatting with users (and helping you become contributors and authors yourselves). Not your first SciPy and feeling "in-your-element"? Make the effort to invite others into the conversations and lunch outings you're having with old friends - we're all here because we care about this stuff.

    take introvert breaks (and be considerate of others who may be doing the same) - I'm an introvert. Though I enjoy interacting with others (one-on-one or in small groups is best for me), it takes a lot of energy and at some point, I run out of steam. That's when I go for a walk, stepping away from the commotion to just have some quiet time.

    be kind to yourself (especially at the sprints) -- Between the tutorials, all of the talks, and the sprints that follow, there will be a flurry of activity. Conferences are already draining enough without trying to get any work done, just meeting a bunch of new people and taking in a lot of information. It took a lot of false starts for me to have productive work output at sprints, but the best thing I've learned about them is to just let go of trying to get a lot done. Instead, try to get something small and well defined done or just help others.

    Stuff to do in Austin

    The conference already has a great list of Things to do in Austin, as well as Restaurants, so I'll just mention a few of my personal favorites.

    Barton Springs Pool. Take a nice dip in the cool waters, and grab a delicious bite from one of the food trucks at The Picnic food truck park.

    Go see the bats. The Congress Ave bridge in Austin is home to the largest urban bat colony in the world. You can read more about this here, but the short version is that around sunset (8-9pm) - a large number of bats stream out from underneath the bridge to go feed on insects. Some days, they leave in waves (this Saturday there were two waves, the first was smaller, but many people left thinking that was the entire show).

    I hope you enjoy SciPy2017!

    read more
  3. June 1st, 2017

    We had another biannual Jupyter team meeting this week, this time it was right nearby in Berkeley. Since I had read a poem at the last meeting, I was encouraged to keep that going and decided to make this a tradition. Here's the result, as delivered this past Friday, recorded by Fernando Pérez (thanks, Fernando!).

    June 1st, 2017

    We struggle -- with ourselves and with each other
    we plan -- we code and write
    the pieces and ideas t'wards what we think is right
    but we may disagree -- about
    the means, about the goals, about the
    shoulders we should stand on --
    where we should stand, what we should stretch toward
    shrink from, avoid, embrace --
    a sense of urgency - but this is not a race
    
    We can't erase the past
    but we have built this future present
    There's much to learn, to do...
    
    Ours not the only path, no one coerced you here
    You chose this -- so did I and here we are --
    still at the barricades and gaining ground
    against the old closed world:
    compute communication comes unshackled
    
    read more
  4. November 9th, 2016

    Two weeks ago, I went down to San Luis Obispo, California for a five day Jupyter team meeting with about twenty five others. This was the first such meeting since my return after being away for two years, and I enjoyed meeting some of the "newer" faces, as well as catching up with old friends.

    It was both a productive and an emotionally challenging week, as the project proceeds along at breakneck pace on some fronts yet continues to face growing pains which come from having to scale in the human dimension.

    On Wednesday, November 9th, 2016, we spent a good chunk of the day at a nearby beach: chatting, decompressing, and luckily I brought my journal with me and was able to capture the poem you will find below. I intended to read it at a local open mic the same evening, but by the time I got there with a handful of fellow Jovyans for support, all of the slots were taken. On Friday, the last day of our meeting, I got the opportunity to read it to most of the larger group. Here's a recording of that reading, courtesy of Matthias Bussonnier (thanks, Matthias!).

    November 9th, 2016

    The lovely thing about the ocean is
    that it
    is
    tireless 
    It never stops
    incessant pendulum of salty foamy slush
    Periodic and chaotic
    raw, serene 
    Marine grandmother clock  
    crashing against both pier
    and rock
    
    Statuesque encampment of abandonment
    recoiling with force
    and blasting forth again
    No end in sight
    a train forever riding forth
    and back
    along a line
    refined yet undefined
    the spirit with
    which it keeps time 
    in timeless unity of moon's alignment
    
    I. walk. forth.
    
    Forth forward by the force
    of obsolete contrition
    the vision of a life forgotten
    Excuses not
    made real with sand, wet and compressed
    beneath my heel and toes, yet reeling from
    the blinding glimmer of our Sol
    reflected by the glaze of distant hazy surf
    upon whose shoulders foam amoebas roam
    
    It's gone.
    Tone deaf and muted by
    
    anticipation
    each coming wave
    breaks up the pregnant pause
    And here I am, barefoot in slacks and tie
    experiencing sensations
    of loss, rebirth and seldom 
    kelp bulbs popping in my soul.
    
    read more
  5. Jupyter's Gravity

    I'm switching jobs.

    For the past two years I've been working with the great team at Disqus as a member of the backend and data teams. Before that, I spent a half-dozen years mostly not working on my thesis at UC Berkeley but instead contributing to to the scientific Python ecosystem, especially matplotlib, IPython, and the IPython notebook, which is now called Jupyter. So when Bloomberg reached out to me with a compelling position to work on those open-source projects again from their SF office, such a tremendous opportunity was hard to pass up. You could say Jupyter has a large gravitational pull that's hard to escape, but you'd be huge nerd. ;)

    I have a lot to catch up on, but I'm really excited and looking forward to contributing on these fronts again!

    read more
  6. indenting with tabs

    2014 04 03 technology

    python

    This post was written as an IPython Notebook. You can view it on nbviewer, or download it.

    Greg Wilson asked on the IPython mailing list:

    Subject: easiest way to insert a literal tab character in a code
    cell? 
    Greg Wilson, on 2014-04-03 18:37,  wrote:
    > Hi,
    > I'd like to put literal tab characters in cells, but of course tab means 
    > "indent" to the editor.  What's the easiest way to do this?
    > Thanks,
    > Greg
    > p.s. because I'm going to write Makefiles in the notebook...
    

    The easiest way to do this is to just get a tab character somewhere that you can copy, and then paste it in.

    In [1]:
    print("\t")
    

    
    
    

    In [2]:
        # I copy pasted the output of the cell above here
    

    An alternative solution is to make a string with tabs and insert it into another cell, using IPython machinery.

    In [3]:
    ip = get_ipython()
    

    In [4]:
    ip.set_next_input("\tMakefiles are awesome")
    

    In []:
        Makefiles are awesome
    

    If you have a file on disk or on the web, you can also just use the %load magic to do this.

    In [5]:
    %load /home/pi/file_with_tabs
    

    In []:
    default:
        cat /etc/issue
        whoami
    

    Such files can be written with the %%writefile cell magic... but of course you need to have inserted tabs there in some manner.

    In [6]:
    %%writefile ~/file_with_tabs
    default:
        cat /etc/issue
        whoami
    

    Overwriting /home/pi/file_with_tabs
    
    

    In [7]:
    !make -f /home/pi/file_with_tabs
    

    cat /etc/issue
    Debian GNU/Linux jessie/sid \n \l
    
    whoami
    pi
    
    

    The more involved, but more usable way

    We can set up CodeMirror to insert tabs instead of spaces.

    In [8]:
    %%javascript
    
    IPython.tab_as_tab_everywhere = function(use_tabs) {
        if (use_tabs === undefined) {
            use_tabs = true; 
        }
    
        // apply setting to all current CodeMirror instances
        IPython.notebook.get_cells().map(
            function(c) {  return c.code_mirror.options.indentWithTabs=use_tabs;  }
        );
        // make sure new CodeMirror instances created in the future also use this setting
        CodeMirror.defaults.indentWithTabs=use_tabs;
    
        };
    

    The reason we attach tab_as_tab_everywhere to IPython is because when we use the %%javascript magic, any variables we define there must be called in the same cell that defined it - they get their own closure. The reason we do this is to allow the notebook javascript to not get screwed up when there are javascript errors. We could have attached it to window or CodeMirror or anything else that's already in javascript-land.

    I covered how to add functions like this to the custom.js file in your profile in my post about disabling blinking in the notebook. That way these little functions are available in every notebook, without you having to insert a cell defining them.

    Now we've got code that allows us to apply the change to all current and future cells. We leave it as an exercise for the interested reader to modify that code and make a little button in the toolbar, to toggle it on a per-cell basis.

    Hints:

    You can get to the code mirror instance via IPython.notebook.get_selected_cell().code_mirror

    This post was written as an IPython Notebook. You can view it on nbviewer, or download it.

    read more
  7. bipython 0.1.0

    bipython
logo

    the boldly indiscriminate python interpreter

    "...because you shouldn't have to choose."

    PROLOGUE

    Two interpreters, both alike in dignity,
    In fair Pythona, where we lay our scene,
    From ancient grudge break to new mutiny,
    Where civil code makes git commits unclean.
    From forth the fatal loins of these two foes
    A newer kind of stranger's given life;
    Whose misadventured piteous overthrows
    Doth with its birth bury its parents' strife.

    ACT I

    Enter bpython and ipython

    bpython

    I'm a fancy terminal-based interface to the Python interpreter. I give you
    inline syntax highlighting and auto-completion prompts as you type, and I'll
    even automatically show you a little tooltip with a docstring and parameter
    list as soon as you hit ( to make the function call, so you always know
    what you're doing! I'm svelte and proud of it - I don't try to do all of the
    shenanigans that ipython does with the shell and the web, but the cool kids
    love my rewind feature for demos. I strive to make interactive python coding
    a joy!

    ipython

    I'm an awesome suite of interactive computing ideas that work together.
    For millennia, I've given you tab-completion and object introspection via
    obj? instead of help(obj) in Python. I also have sweet shell features,
    special magic commands (%run, %timeit, %matplotlib, etc.) and a
    history mechanism for both input (command history) and output (results
    caching).

    More recently, I've decoupled the REPL into clients and kernels, allowing
    them to run on independent of each other. One popular client is the
    IPython Notebook which allows you to write code and prose using a web
    browser, sending code to the kernel for execution and getting rich media
    results back inline. The decoupling of clients and kernels also allows
    multiple clients to interact with the same kernel, so you can hook-up to
    that same running kernel from the terminal. The terminal workflow makes
    more sense for some things, but my user interface there isn't as polished
    as bpython's.

    Enter bipython

    bipython

    By your powers combined... I am bipython!

    Exeunt

    The Power is Yours!

    pip install  bipython
    easy_install bipython
    

    bipython requires ipython, pyzmq, bpython, and urwid.

    For now, you'll need to have a running ipython kernel before running bipython. You can do this by either opening a notebook or running ipython console. It won't always be like this, I'll fix it as soon as I can, but it'll be sooner with your help over ivanov/bipython.

    After that, just run bipython and enjoy the ride.

    Here's a walkthrough of ipython, bpython, and bipython:

    The screencast is 20 minutes long, but here I'll play it back double speed. There's no sound, and you can pause at any time and select / copy portion of the text as you like. Changing the browser font size in the usual way works, too. (click here if the embed didn't work)

    read more
  8. stem-and-leaf plots in a tweet


    Summary: I describe stem plots, how to read them, and how to make them in Python, using 140 characters.

    My friend @JarrodMillman, whose office is across the hall, is teaching a computational statistics course that involves a fair amount programming. He's been grading these homeworks semi-automatically - with python scripts that pull the students' latest changes from GitHub, run some tests, spit out the grade to a JSON file for the student, checks it in and updates a master JSON file that's only accessible to Jarrod. It's been fun periodically tagging along and watching his suite of little programs develop. He came in the other day and said "Do you know of any stem plot implementation in python? I found a few, and I'm using one that's ok, but it looks too complicated."

    For those unfamiliar - a stem plot, or stem-and-leaf plot is a more detailed kind of histogram. On the left you have the stem, which is a prefix to all entries on the right. To the right of the stem, each entry takes up one space just like a bar chart, but still retains information about its actual value.

    So a stem plot of the numbers 31, 41, 59, 26, 53, 58 looks like this:

     2|6
     3|1
     4|1
     5|389
    

    That last line is hard to parse for the un-initiated. There are three entries to the right of the 50 stem, and these three entries 3 8 and 9 is how the numbers 53, 58, and 59 are concisely represented in a stem plot

    As an instructor, you can quickly get a sense of the distribution of grades, without fearing the binning artifact caused by standard histograms. A stem-plot can reveal subtle patterns in the data that are easy to missed with usual grading histograms that have a binwidth of 10. Take this distribution, for example:

    70:XXXXXXX
    80:XXXXXXXXXXX
    90:XXXXXXX
    

    Below are two stem plots which have the same profile as the above, but tell a different story:

     7|7888999
     8|01123477899
     9|3467888
    

    Above is a class that has a rather typical grade distribution that sort of clumps together. But a histogram of the same shape might come from data like this:

     7|0000223
     8|78888999999
     9|0255589
    

    This is a class with 7 students clearly struggling compared to the rest.

    So here's the code for making a stem plot in Python using NumPy. stem() expects an array or list of integers, and prints all stems that span the range of the data provided.

    from __future__ import print_function
    import numpy as np
    def stem(d):
        "A stem-and-leaf plot that fits in a tweet by @ivanov"
        l,t=np.sort(d),10
        O=range(l[0]-l[0]%t,l[-1]+11,t)
        I=np.searchsorted(l,O)
        for e,a,f in zip(I,I[1:],O): print('%3d|'%(f/t),*(l[e:a]-f),sep='')
    

    Yes, it isn't pretty, a fair amount of code golfing went into making this work. It is a good example for the kind of code you should not write, especially since I had a little bit of fun with the variable names using characters that look similar to others, especially in sans-serif typefaces (lI10O). Nevertheless, it's kind of fun to fit much functionality into 140 characters.

    Here's my original tweet:

    You can test it by running it on some generated data:

    >>> data = np.random.poisson(355, 113)
    >>> data
    array([367, 334, 317, 351, 375, 372, 350, 352, 350, 344, 359, 355, 358,
       389, 335, 361, 363, 343, 340, 337, 378, 336, 382, 344, 359, 366,
       368, 327, 364, 365, 347, 328, 331, 358, 370, 346, 325, 332, 387,
       355, 359, 342, 353, 367, 389, 390, 337, 364, 346, 346, 346, 365,
       330, 363, 370, 388, 380, 332, 369, 347, 370, 366, 372, 310, 348,
       355, 408, 349, 326, 334, 355, 329, 363, 337, 330, 355, 367, 333,
       298, 387, 342, 337, 362, 337, 378, 326, 349, 357, 338, 349, 366,
       339, 362, 371, 357, 358, 316, 336, 374, 336, 354, 374, 366, 352,
       374, 339, 336, 354, 338, 348, 366, 370, 333])
    >>> stem(data)
     29|8
     30|
     31|067
     32|566789
     33|00122334456666777778899
     34|02234466667788999
     35|001223445555577888999
     36|12233344556666677789
     37|0000122444588
     38|0277899
     39|0
     40|8
    

    If you prefer to have spaces between entries, take out the sep='' from the last line.

    >>> stem(data)
     29| 8
     30|
     31| 0 6 7
     32| 5 6 6 7 8 9
     33| 0 0 1 2 2 3 3 4 4 5 6 6 6 6 7 7 7 7 7 8 8 9 9
     34| 0 2 2 3 4 4 6 6 6 6 7 7 8 8 9 9 9
     35| 0 0 1 2 2 3 4 4 5 5 5 5 5 7 7 8 8 8 9 9 9
     36| 1 2 2 3 3 3 4 4 5 5 6 6 6 6 6 7 7 7 8 9
     37| 0 0 0 0 1 2 2 4 4 4 5 8 8
     38| 0 2 7 7 8 9 9
     39| 0
     40| 8
    

    To skip over empty stems, add e!=a and in front of print. This will remove the 300 stem from the output (useful for data with lots of gaps).

    >>> stem(data)
     29| 8
     31| 0 6 7
     32| 5 6 6 7 8 9
     33| 0 0 1 2 2 3 3 4 4 5 6 6 6 6 7 7 7 7 7 8 8 9 9
     34| 0 2 2 3 4 4 6 6 6 6 7 7 8 8 9 9 9
     35| 0 0 1 2 2 3 4 4 5 5 5 5 5 7 7 8 8 8 9 9 9
     36| 1 2 2 3 3 3 4 4 5 5 6 6 6 6 6 7 7 7 8 9
     37| 0 0 0 0 1 2 2 4 4 4 5 8 8
     38| 0 2 7 7 8 9 9
     39| 0
     40| 8
    

    Thanks for reading.

    read more
  9. disabling blinking

    2014 01 29 technology

    python vision

    Background: Text editing in the IPython Notebook is provided by an excellent JavaScript-based CodeMirror text editor. This might be more detail than you want, but I'm a vision scientist so I hope you'll indulge me.

    The cursor is meant to tell the user the current location.

    The human visual system has a pre-cortical lag of roughly 50-90 ms (read more about P1).

    That's how long it takes from something changing on the screen to cause an avalanche of photons to barrel towards your eyeball, be phototransduced and processed by several stages of cells in the retina, finally causing retinal ganglion cells to fire an action potential down their axons through the optic nerve, make its way to a processing relay station called the LGN, with those cells firing action potential down their axons, with those spikes finally ending up in the primary visual cortex.

    By ~150 ms, our brains have processed the visual input enough to perform a non- trivial ammount of object recognition.

    The default blink rate for CodeMirror is 530ms.

    That's as slow as molasses in January!

    Say that I've been distracted and looked away from the screen. When I look back at the scree, half of the time it will take 3 times longer for me to get the information that I want ("where's my cursor") than if that cursor was clearly visible at all times. Now it's not always that bad, because sometimes my gaze will land on the screen and even though the cursor isn't visible, it appears in a few milliseconds, and so it takes just as long as if the cursor was there the whole time. But if I happen to be particularly unlucky (there's a reason I don't gamble), it can take 6 times longer.

    Try it out

    Here's the bit of JavaScript code you need to disable blinking in CodeMirror.

    CodeMirror.defaults.cursorBlinkRate=0
    

    If you type that into the JavaScript console of your webbrowser, that setting will apply to all cells created in the current IPython Notebook. If you don't know how to open your browser's Javascript console, don't frett, just make a new cell with just the following lines in there, execute it, and make a new cell to see how you like it.

    %%javascript
    CodeMirror.defaults.cursorBlinkRate=0
    

    Make the change stick

    IPython has a notion of profiles to allow for different kinds of configurations. If this is news to you, you've probably just been using the default profile and not known it. In the shell, run the ipython profile create command to be sure (don't worry, if you alreay have a profile, this won't overwrite it). Now ipython locate profile will tell you the directory which contains all of the configuration for the default profile.

    In [1]:
    !ipython profile create
    

    In [2]:
    !ipython locate profile
    

    /home/pi/.ipython/profile_default
    

    In [3]:
    x = !ipython locate profile
    

    In [4]:
    cd $x.s
    

    /home/pi/.ipython/profile_default
    

    In [5]:
    ls
    

    db/  history.sqlite  history.sqlite-journal  ipython_config.py  ipython_nbconvert_config.py  ipython_notebook_config.py  log/  pid/  security/  startup/  static/
    

    There's a lot of stuff there, but we just need to add our one line to the end of the file in static/custom/custom.js

    In [6]:
    cd static/custom/
    
    /home/pi/.ipython/profile_default/static/custom
    

    In [7]:
    ls
    

    custom.css  custom.js
    

    In [8]:
    !echo "codemirror.defaults.cursorblinkrate=0" >> custom.js
    

    "I want it all and I want it now!"

    You say you don't want to save your current notebook and reload it to get the updated CodeMirror settings? You just want all cells in the current notebook to change their behavior? Well, OK, Freddie:

    In [9]:
    %%javascript
    var rate = 0;
    // apply setting to  all current CodeMirror instances
    IPython.notebook.get_cells().map(
        function(c) {  return c.code_mirror.options.cursorBlinkRate=rate;  }
    );
    
    // make sure new CodeMirror instance also use this setting
    CodeMirror.defaults.cursorBlinkRate=rate;
    

    I hope you enjoyed this little IPython customization detour. If you want more information about how to get rid of blinking in other programs you use every day, here is an invaluable resource on that matter.

    Remember, blinking in user interfaces is bad, but blinking in vision is very important.

    This post was written as an IPython Notebook. You can view it on nbviewer, or download it

    read more

Page 1 / 2 »

social