Ben Bakelaar

Highland Park, NJ

Category: Technology

Project management with LiquidPlanner and kanban board

I don’t know about you, but project management systems don’t seem to do a very good job of giving you the “big picture” view! This is the primary “Projects” view in LiquidPlanner with 29 active projects. Taking some cues from kanban boards for software development cycles, which my colleague introduced me to here at my new job, I designed a visual “overview” system to surface, at the highest level, which projects were active and which were “marked active” but not actually active in a given timeframe (i.e. week).

Screen Shot 2016-03-29 at 2.57.23 PM
Converting to post-it notes with an initial hierarchy
kanban-liquid-planner-project-management-01

Testing out assigning projects to the categories
kanban-liquid-planner-project-management-02

Moved to a more central wall, with blue tape for boundaries
kanban-liquid-planner-project-management-03
Final iteration with critical/needs attention (!), in motion/active (|>), on hold (| |), and done ([ ])kanban-liquid-planner-project-management

Exporting BitBucket issue queue and converting JSON to CSV

I recently came across this need in my new job. With an active queue of 122 issues for a particular repository, I needed to get the data out of the BitBucket UI which only shows 25 rows at a time, and not all columns (since there several dozen!).

The first issue was that you need to be a full admin on the repository in order to export. Since I didn’t have that permission, nor did I particularly want it, I had to request an export from the lead developer.

https://confluence.atlassian.com/bitbucket/export-or-import-issue-data-330797432.html

Repository administrators can export issues out of or import issues into a Bitbucket Cloud issue tracker.  The feature is useful for moving issues between two repositories.

I then received a nice shiny ZIP file that contained… a JSON file. While JSON is great, it is a machine-readable format, not human readable. And much like XML, the content is kind of split out across content types, rather than your standard CSV format where you have 1 row with all associated information. I opened the JSON file in TextWrangler (working on a Mac here) and was presented with this.

Screen Shot 2016-03-29 at 1.15.13 PM

One single line, 1.3mb file. Doing a text wrap didn’t help too much.

Screen Shot 2016-03-29 at 1.16.53 PM

Without much additional analysis, I figured I could just find a JSON to CSV converter. After googling “json to csv”, I tried the top 3 results.

Screen Shot 2016-03-29 at 1.18.03 PM

KONKLONE.IO/JSON

The first one, http://konklone.io/json, presents a nice clean interface. Paste your JSON in the top, and it will parse it into a table below. Unfortunately, the 1.3MB of text was just too big for it. Maybe if I had a newer computer with more RAM, but even then I bet not.

Screen Shot 2016-03-29 at 1.24.00 PM

CONVERTCSV.COM/JSON-TO-CSV

The 2nd converter offers either copy/paste, URL loading, or file upload. Unfortunately, all I got was “Field1” as a result after I uploaded. So I quickly moved on to the third option.

Screen Shot 2016-03-29 at 1.52.40 PM

JSON-CSV.COM

The third result looked promising, nice and clean interface, and it actually spit back a table that looked like it might be the one! Alas, my file was 1.22MB and so I would need to upgrade to get the file.

Screen Shot 2016-03-29 at 1.54.00 PM

My next inclination was to just try to chop off the end of the file. However, that’s not so easy in an XML/JSON file that is essentially just one long string of text! You essentially have to decode the JSON structure, and being more familiar with XML, I was a bit lost. I knew there was some internal structure, but I needed to visually see it.

{“milestones”:[],”attachments”:[{“path”:”attachments\/2c30fb3767644898a9d0fbf4b08e56a1″,”issue”:411,”user”:”TonyaG”,”filename”:”Webform Date Field 2.png”},{“path”:”attachments\/c018dd7e20c24a2cb7b47142f994c37b”,”issue”:411,”user”:”TonyaG”,”filename”:”Webform Date Field 1.png”},{“path”:”attachments\/76b77eab038946b080e536973c5cce54″,”issue”:420,”user”:”ahepler”,”filename”:”Screen Shot 2016-03-24 at 11.07.19 AM_thumb.png”},{“path”:”attachments\/0fe295bc6f4949f28e2fe9b33ee9f9b0″,”issue”:420,”user”:”ahepler”,”filename”:”Screen Shot 2016-03-24 at 11.05.53 AM_thumb.png”},{“path”:”attachments\/3b15832709064ae583eee62f04c702e2″,”issue”:420,”user”:”ahepler”,”filename”:”Screen Shot 2016-03-24 at 11.05.16 AM_thumb.png”},{“path”:”attachments\/10eaeead075d42aea9f0a9e0e13cf6e7″,”issue”:377,”user”:”amailyan”,”filename”:”Screen Shot 2016-02-25 at 12.23.58 PM.png”},{“path”:”attachments\/0f7153ecaf364af98748d1279c2aa8fc”,”issue”:398,”user”:”jlongia”,”filename”:”Screen Shot 2016-01-07 at 9.32.07 AM.png”},{“path”:”attachments\/e14388a3a27b4397aa6ff3fe03a60ebf”,”issue”:393,”user”:”amailyan”,”filename”:”Screen Shot 2015-12-11 at 12.01.42 PM.png”},{“path”:”attachments\/a410852c9be94ea2b65c698e173cf53e”,”issue”:384,”user”:”itmaybejj”,”filename”:”screengrab 2015-12-02 at 11.59.07 AM.png”},{“path”:”attachments\/501840a4428a45b8a14108b310e34cbd”,”issue”:355,”user”:”byronveale”,”filename”:”globalhealth_explore_box_inspected.png”},{“path”:”attachments\/c8d93d3807d2433e97affe46c2080927″,”issue”:355,”user”:”byronveale”,”filename”:”chw_explore_box_inspected.png”},{“path”:”attachments\/d5e34376bf2b4622be05fbab1e7d23eb”,”issue”:355,”user”:”byronveale”,”filename”:”global_health_coming_from_css_injector_use_uuid_instead_of_paneid.png”},{“path”:”attachments\/f5664e878a2f4a48a71fbec5888ab3b4″,”issue”:355,”user”:”byronveale”,”filename”:”chw_coming_from_css_injector_use_uuid_instead_of_paneid.png”},{“path”:”attachments\/9740c19f20874ea78830fc896ac3c5e0″,”issue”:355,”user”:”byronveale”,”filename”:”chw_explore_box_looks_okay_as_admin.png”},{“path”:”attachments\/b175971e0d21424b9ce134d2553d7510″,”issue”:355,”user”:”byronveale”,”filename”:”chw_explore_box_looks_broken_as_editor.png”},{“path”:”attachments\/842917d280ca4095b4072dea0eb3295a”,”issue”:355,”user”:”byronveale”,”filename”:”this_div_and_h2_above_lose_the_cascaded_id.png”},{“path”:”attachments\/8139b5b5038d45728e32538a0c762943″,”issue”:355,”user”:”byronveale”,”filename”:”this_div_gets_id_from_page_customization_option.png”},{“path”:”attachments\/f75cb969fe7a4d2887c9328d58b744e8″,”issue”:355,”user”:”byronveale”,”filename”:”editors_cant_customize_so_no_idz_for_them.png”},{“path”:”attachments\/292d1333ac6b4f438b97eb60765513da”,”issue”:297,”user”:”TonyaG”,”filename”:”thumbnail.jpg”},{“path”:”attachments\/354e30bd312844e781090a8a834c20fa”,”issue”:223,”user”:”byronveale”,”filename”:”there_is_no_content_to_be_shown.png”},{“path”:”attachments\/0220b2f6060342b3a522a023530e68e2″,”issue”:93,”user”:”mdmuzzie”,”filename”:”submission-confirm.png”},{“path”:”attachments\/d43975d64a8a44078c166478aafbf150″,”issue”:106,”user”:”jmoraca”,”filename”:”dripp-eventcolumn.PNG”},{“path”:”attachments\/33969cffc16d4d79bc4eebd847794ebb”,”issue”:125,”user”:”jmoraca”,”filename”:”dripp-addcontent.PNG”},{“path”:”attachments\/860b8de7f5054b3e9f95e2b29f764ef8″,”issue”:174,”user”:”mdmuzzie”,”filename”:”Screen Shot 2014-08-13 at 11.51.00 AM.png”},{“path”:”attachments\/feaf3d78979d47a78c4d7abfbffbd3b6″,”issue”:123,”user”:”jmoraca”,”filename”:”dripp-courses-block-fields.PNG”},{“path”:”attachments\/ba0706ffa8ca4424a140bed3ce4b547a”,”issue”:150,”user”:”mdmuzzie”,”filename”:”Screen Shot 2014-07-15 at 5.17.14 PM.png”},{“path”:”attachments\/cf0c3643796a436187f2efe248da6ddd”,”issue”:209,”user”:”jmoraca”,”filename”:”pwds-savebasivpage.PNG”},{“path”:”attachments\/122c62dcf17040ca9e9c1a6b767b275a”,”issue”:185,”user”:”mdmuzzie”,”filename”:”custom permissions.png”},{“path”:”attachments\/fff8ef23e670443288d616545fc85235″,”issue”:210,”user”:”jmoraca”,”filename”:”pwds-dropdowns.PNG”},{“path”:”attachments\/cf0971756ab540f18ca5b4a7d27cd52d”,”issue”:193,”user”:”jmoraca”,”filename”:”pwds-repeatevents-edit.PNG”},{“path”:”attachments\/61a228e18bb041d3a39cc3501945190a”,”issue”:193,”user”:”jmoraca”,”filename”:”pwds-repeatevents.PNG”},{“path”:”attachments\/ae9fbb06e13c4172979aabfacb990882″,”issue”:190,”user”:”jmoraca”,”filename”:”pwds-peoplelist.jpg”},{“path”:”attachments\/dbd4b3b57d28438da870fb4d30fbd1ef”,”issue”:94,”user”:”mdmuzzie”,”filename”:”theme_dev-dev.png”},{“path”:”attachments\/74eddc15055f4d6896c43bd624fe5816″,”issue”:91,”user”:”mdmuzzie”,”filename”:”bobst-contact.png”},{“path”:”attachments\/f83141bbb58b4f0c9ed61b689f90bea9″,”issue”:98,”user”:”mdmuzzie”,”filename”:”overlap.png”},{“path”:”attachments\/a152cdd1e868416a89fe87b5ba5ae34e”,”issue”:71,”user”:”jmoraca”,”filename”:”news-incorrectmenu.PNG”},{“path”:”attachments\/5f1160a2eecc4ad9b879751742f69e0f”,”issue”:107,”user”:”mdmuzzie”,”filename”:”bronies.png”}],”versions”:[{“name”:”1.1 – Honey Badger”},{“name”:”1.2 – Grumpy Cat”},{“name”:”1.3 – Frisky Dingo”},{“name”:”1.4 – Elated Elephant”},{“name”:”1.5 – Dubious Fox”},{“name”:”1.6 – Creative Gecko”},{“name”:”1.x”},{“name”:”2.x”}],”comments”:[{“content”:”@mdmuzzie Excellent point. I think the base theme should have this. “,”created_on”:”2016-03-29T13:29:04.209346+00:00″,”user”:”jmoraca”,”updated_on”:null,”issue”:421,”id”:26571412}

asdf

Running WordPress on AWS EC2 Ubuntu micro-tier with WP-CLI

I’ve just moved my site off of Rackspace to AWS EC2 after having been able to experiment with EC2 at work for the past several months. The biggest difference I note is with SSD drives serving the database and files – this offers a lightning-speed improvement! A micro-tier server (1 CPU, 1 GB RAM) should run around $20 or less per month, metered at an hourly rate for 750 hours per month.

These are the articles I had to reference when migrating my site from Rackspace Linux hosting to AWS EC2. There are a few “gotchas” and caveats with EC2 that make it best suited for technical folks (programmers, server admins, etc.). You need to be good with the command line – what you get is “dedicated” hosting, relative to shared tenancy hosting like BlueHost and GoDaddy, that is so much faster, for almost the same price!

AWS EC2 Ubuntu setup for WordPress (in reverse order, start at the bottom)
—–

Regenerate thumbnails
http://www.wpbeginner.com/plugins/regenerate-thumbnails-new-image-sizes-wordpress/

Amazon EC2 WordPress Not Cropping or Resizing Images Fix
https://wordimpress.com/amazon-ec2-wordpress-not-cropping-or-resizing-images-fix/
sudo apt-get install php5-gd

Increase file upload size
https://forums.aws.amazon.com/thread.jspa?threadID=90905

Using WP-CLI
http://torquemag.io/using-wp-cli-to-set-up-a-test-version-of-your-site/

THE BEST THING ever!
http://wp-cli.org/commands/search-replace/

Enable mod_rewrite
http://jafty.com/blog/enable-mod_rewrite-on-apache-ec2-linux-server/

Enabling permalinks
https://codex.wordpress.org/Using_Permalinks

Locating user name running Apache (www-data)
http://unix.stackexchange.com/questions/41241/how-to-check-which-apache-group-i-can-use-for-the-web-server-to-write

Create new user in MySQL and give it full access to one database
http://stackoverflow.com/questions/1720244/create-new-user-in-mysql-and-give-it-full-access-to-one-database

How To Install WordPress On Amazon EC2 Ubuntu Instance
http://www.comtechies.com/2012/12/how-to-install-wordpress-on-amazon-ec2.html

Email overload part 1

This is a post in progress. 

I took some time at the beginning of this week to take a look at the state of my inbox. As my career in technology has progressed, I’ve become interested in reflecting on email activity as a proxy or gauge for the various aspects of my job. Below I’ll describe the context surrounding these numbers, my current strategy for inbox management, and some details of my inbox folder structure.

inbox-02

 

In the past, I’ve always ended up using my inbox as a sort of to-do list, and filing messages once they are completed. This is of course a fairly inefficient way of doing things, but in my environment at a state university, there are some factors which lend themselves to this approach: 1) you may find yourself needing to look up messages from several years ago, 2) you may need to trace the chain of events through a particular chain of emails or produce a report on a certain time period of a certain project, and 3) there is a neverending pile of projects and tasks that get added, while most pre-existing projects and tasks linger and never get “finished”. In this environment, traditional systematic approaches to PM and task management fail because there is not enough time to devote to keeping every item in a system, or workflow, or queue.

Having started my current job in July 2013, I had done a pretty good job of managing my inbox and filing emails for the first 12 months, but as projects started piling up I began to fall behind – a few vacations and conferences later, I was permanently behind. So I created another layer to my inbox – the current week of emails would sit in the inbox, while the past 3 weeks would each get filed into their own folder. I set the folder to display the total number of items, and when I have downtime I work backwards, starting at 3 weeks ago and reviewing which emails can now be filed, which are now resolved, and which still need action taken. On a good week, I can get the prior 3 weeks of emails down to about 50 or so, split between the 3 folders. On a bad month, I haven’t had any time to file and there will be ~1000 emails between the 4 most recent weeks. This of course takes a concerted effort to sort through, and feels overwhelming – but not as overwhelming as having a single inbox with thousands of messages, some unread, some flagged, all unfiled… so it works.

Describing the chart above, I took this count on Monday AM, so there were only 19 emails that had come in. The prior week only had 124 unfiled, almost 300 from 2 weeks ago, and less than 100 from 3 weeks ago. By the end of that Monday, I had gotten 3 weeks ago down to 10-20. The Flagged Recent shows there were 54 emails flagged out of the ~500 in the past 4 weeks. The Flagged Old showed there were 154 emails that remained flagged from prior to 4 weeks ago. At a certain point I do just let them go, but again working in a slowly moving state job, there are no shortage of good ideas and action items that need follow-up, but it might be 4-6 months before it is possible or appropriate to follow up. I suspect most people would simply let these emails fall by the wayside, at the most waiting to be reminded of the issue by a colleague, boss, or report.

The remainder of the folders in the chart reflect folders that have emails filed in them since July 2013, so almost 18 months. This gives me a way to gauge how much activity is happening in a certain area of my job. For example, Hadoop, a small side project that wasn’t even on my list of responsibilities ended up generating 2500 emails in 18 months – clearly showing the need for extra resources, which we finally obtained in the past month in the form of a consultant.

© 2017 Ben Bakelaar

Theme by Anders NorenUp ↑