Daily Technology2023-03-23T17:49:27+00:00http://dailytechnology.net/Brian Daileybrian@realm3.comAdjusting the HHKB HASU (mod) Controller Layout2021-09-22T00:00:00+00:00http://dailytechnology.net//programming/2021/09/22/remapping-hasu-controller<p>I recently acquired an HHKB Professional 2 with the HASU bluetooth controller
mod. I did not find documentation for remapping the keys using tmk or qmk in
one place, but I did cobble the information together and figured I would write
it up here. It’s more for my own reference than anything else, but perhaps it
will be useful to someone else!</p>
<p>This is assuming that you’re using a Mac. You’ll need homebrew installed and
you’ll need to use your laptop keyboard (or another spare) to flash the changes
to the HHKB.</p>
<p>Install <code class="language-plaintext highlighter-rouge">dfu-programmer</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brew install dfu-programmer
</code></pre></div></div>
<p>Go to the <a href="http://www.tmk-kbd.com/tmk_keyboard/editor/unimap/?hhkb_rn42">TMK keymap editor for the HHKB Pro1/Pro2 Controller Bluetooth</a>
and make your changes and download the hex file.</p>
<p>Put your keyboard into flashing mode by either pushing the red button on the
back or pressing the combination <code class="language-plaintext highlighter-rouge">Left Shift</code> + <code class="language-plaintext highlighter-rouge">Right Shift</code> + <code class="language-plaintext highlighter-rouge">Fn</code> + <code class="language-plaintext highlighter-rouge">P</code>.
After doing this the keyboard should not respond to further key presses, so
continue using another keyboard but do not disconnect it.</p>
<p>Run the following command in Terminal, and you should see similar output.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ dfu-programmer atmega32u4 erase
Checking memory from 0x0 to 0x6FFF... Not blank at 0x1.
Erasing flash... Success
Checking memory from 0x0 to 0x6FFF... Empty.
</code></pre></div></div>
<p>Next, apply the hex file you downloaded earlier from the keymap editor.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ dfu-programmer atmega32u4 flash <path_to_.hex>
Checking memory from 0x0 to 0x6FFF... Empty.
0% 100% Programming 0x7000 bytes...
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] Success
0% 100% Reading 0x7000 bytes...
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] Success
</code></pre></div></div>
<p>Lastly, run this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>dfu-programmer atmega32u4 reset
</code></pre></div></div>
<p>Your changes should now be ready to use!</p>
Remapping Caps Lock in Multiple Operating Systems2021-09-09T00:00:00+00:00http://dailytechnology.net//programming/2021/09/09/remapping-caps-lock-to-ctrl<p>My colleague <a href="https://www.linkedin.com/in/barry-archer-pmp-csm-886b9749/">Barry Archer</a> recently sold me an HHKB (<a href="https://hhkeyboard.us/">“Happy Hacking Keyboard”</a>).
It’s a little different than most other keyboards I’ve used (and I’m a fan of
either 60% of 75% layouts). For example, the Backspace key is below the number
row, and the backslash key is in the number row. I found I was able to adjust to
this and back to the “Normal” positioning fairly quickly after some practice.
For example, if I needed to use my laptop keyboard for a while, readjusting my
fingers only took one or two mistakes. The brain’s plasticity never ceases to
surprise me.</p>
<p>However, one other change on this layout was harder for me to adjust to. In
particular, there is no left “Ctrl” key. Instead, you will find it where you
might normally find a Caps Lock key. This left me SHOUTING UNNECESSARILY when I
was using a more standard keyboard layout. I use the Ctrl key for navigating words,
particularly on the command line, and I kept tripping up on this change.</p>
<p>Since I try not to shout much on the internet, Caps Lock isn’t super useful for
me.<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> It makes sense to remap it to something more useful, like the Control
key, across all my keyboards. However, when I set out to do this I found that
there is a ton of out-dated information floating around (registry hacks,
anyone?). Hence, I hope it might be useful to see this in one place for others.
These tricks worked, at least, in September 2021.</p>
<ul>
<li>For <strong>Ubuntu Linux</strong> (20.04 or newer, but also probably older version as well),
install the Gnome tweaks tool. You can do this by opening a
terminal and running <code class="language-plaintext highlighter-rouge">sudo apt install gnome-tweaks</code>. From there, open up
“Gnome Tweaks” and simply navigate to the keyboard preferences panel. From
there you can easily map Caps Lock to Ctrl.</li>
<li>For <strong>Mac OS</strong>, open up System Preferences and then “Keyboard.” Click the
“Modifier Keys” button. From here you can remap Caps Lock to Control. Note
that if you use an external (USB) keyboard you will need to remap this for
each keyboard.</li>
<li><strong>Microsoft Windows</strong>, use the <a href="https://docs.microsoft.com/en-us/windows/powertoys/">PowerToys</a>
utility. Once it’s installed, open it and click “Keyboard Preferences.” From
there you can remap Caps Lock to Ctrl. This works on Windows 10 or Windows 11.</li>
</ul>
<p>That covers all of the major operating systems. Happy keyboarding!</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>I tried the <a href="https://colemak.com/">colemak</a> layout for a year or so and one of the things I liked about it was that it remapped Caps Lock to backspace. However, I found that my typing error rate never recovered so I eventually went back to Qwerty and continued to ignore the Caps Lock key. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Ten Years at Stratasan: A Retro2021-04-26T00:00:00+00:00http://dailytechnology.net//management/2021/04/26/ten-years-at-stratasan<p>In May of 2021, it will be ten years since we started Stratasan. Even though the
Stratasan adventure is far from over, it seems as good a time as any to write up
a retrospective.</p>
<h3 id="wait-whats-a-stratasan">Wait… what’s a Stratasan?</h3>
<p>Stratasan is a company I co-founded with Jason Moore in 2011. <sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> We help
healthcare companies (usually acute care hospitals) see what’s going on in
their market. Where should they build outpatient clinics? What specialists
should they hire, given demographic changes in their market? We also work to
collect healthcare data from facilities to report to various entities (e.g., a
state Department of Health or a hospital association) and to use for
business intelligence.</p>
<h3 id="was-it-what-i-expected">Was it what I expected?</h3>
<p>Yes and no. When Jason asked if I would come on full time, I said two things: I
saw it as a great opportunity to learn some new things, and I didn’t figure
I would be a good fit for technical leadership once the company grew to more
than six or so employees.</p>
<p>The former was accurate! The latter remains to be seen. We’re at about
seventy people now and I still feel like I’m contributing value. But, of
course, I <em>would</em> say that, wouldn’t I?</p>
<h3 id="what-do-you-know-how-that-you-wish-youd-known-then">What do you know how that you wish you’d known then?</h3>
<h4 id="management-is-its-own-discipline-worthy-of-study-itself">Management is it’s own discipline, worthy of study itself.</h4>
<p>In the beginning, I wrote a lot of code. Pretty quickly, I hired another
engineer (the very capable Matt George) and worked alongside someone who was a
much better developer than myself. Over a few short years, we built a crack team
of engineers that included some top-caliber folks like Jason Myers and Will
Golden. During that time, I still saw myself as a developer first, and a team
lead secondly. I considered the term “manager” as a pejorative and thus did not
feel it was necessary for me to educate myself on what it <em>meant</em> to be a
manager. After all, I had a degree in Business Administration, so I already knew
what that meant, right?</p>
<p>Wrong. Management is it’s own discipline. It took me probably four to five years
to properly understand just how different that discipline was from the craft of
software development. When I say “management” I don’t mean “business
management”. I mean it in the Andrew Grove sense of the term: being responsible
for growing the output of my team. Manager as enabler: removing impediments
(often in the form of bad processes), keeping the team healthy, etc.</p>
<p>I still feel that having <em>some</em> technical expertise is helpful if you’re going
to be responsible for the technical side of a product company, but it’s less
important than the ability to solve problems (a talent most developers have in
spades!), have healthy and honest conversations, and communicate a vision that
everyone can be excited about.</p>
<h4 id="product-management-is-hard">Product management is hard.</h4>
<p>When you start a company, you wear <em>all</em> of the hats. As you hire people, you
get to hand off some of those hats to folks who do that job well. Among the many
hats I’ve worn (developer, security guy, IT support, compliance expert,
reader-of-contracts…) product manager is one of the hardest. Being a product
manager is a mess of vision casting, strategic planning, data analysis, and
domain expertise. Furthermore, it is a company nexus that sits between sales,
development, and the C-suite’s future vision. It has tension with everyone because you have to
balance capacity with endless ideas. Someone has to be told “no.” Not everyone
takes it well.</p>
<p>We still have a lot of work to do on this front. The hardest thing about a
growing company (even one that is growing organically and relatively steadily)
is that the needs of the day are <em>constantly</em> changing. The minute you think
you’ve gotten the organizational structure in a sweet spot, something new
happens.</p>
<h2 id="would-you-do-it-again">Would you do it again?</h2>
<p>Absolutely. But only with the exact same people.</p>
<p>I have said this many times in person, but the two things that I’ve enjoyed the
most about the whole ride are, in this order:</p>
<ol>
<li>The people I have gotten to work with.</li>
<li>The problems I get to solve.</li>
</ol>
<p>There’s a reason I still list “Problem Solving” as my job title on LinkedIn.
It’s what got me into software development, and why I’m still here.</p>
<h3 id="other-important-things">Other Important Things</h3>
<h4 id="shared-values-are-paramount">Shared values are paramount.</h4>
<div class="pull-right img text-center"><img src="/img/2021-04-26-jmo-brian-small.png" class="img-circle" /><br /><small>Jason & Brian @ Stratasan's 8th Anniversary</small></div>
<p>Jason deserves most of the credit for Stratasan’s success. What convinced me to
work with Jason was not his winsome ways (although they are pretty winsome!). It
was his shared values. It’s what made our relationship work not just in the
short term while we’re figuring things out, but also in the long term: even in
the valley of entrepreneurial despair.</p>
<p>Jason’s core values have a significant impact on <em>Stratasan’s</em> core values. It
was important that <em>his</em> values align with the entire leadership team, including
mine. Otherwise there will be irreconcilable differences in how we think about
business.</p>
<h4 id="the-leadership-team-benefits-from-having-technical-and-product-expertise">The leadership team benefits from having technical and product expertise.</h4>
<p>Having a technical expert in leadership is important. (Of course, I <em>would</em> say
that, wouldn’t I?)</p>
<p>Having someone with even a modicum of technical background in the same room as
the board has been helpful for us. Being able to hear about concerns, trends,
and strategies straight from the investors is critical. Companies are always
talking about organizational <em>alignment</em> - and for good reasons! If you’re a product company, you
should put your product people in the same room with your business operations
and domain experts. Why? Because product people are great at problem solving.
Often your investors will be most familiar with the business problems at hand (that’s
why they’re willing to put up money and embrace the risk of backing a startup!).
The less layers you have between your top technical/product leadership and the board, the
better.</p>
<h3 id="what-does-the-future-hold">What does the future hold?</h3>
<div class="pull-right img text-center"><img src="/img/2021-04-26-jmo-brian-2-small.png" class="img-circle" /><br /><small>We're already a much larger group two years after this photo!</small></div>
<p>I wish I knew! When we started the company, I told Jason that I probably
wouldn’t stick around for very long. I assumed that by the time we hit six
employees, I would probably be doing something else. Ten years later and seventy
people and I’m still loving every minute of it. I can’t wait to see what the
next decade looks like!</p>
<p>–</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>There were actually three co-founders (the third being Tod Fetherling) but Tod exited the company fairly early to start Perception Health. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Cultivating a Culture of Partnership2019-11-19T00:00:00+00:00http://dailytechnology.net//business/2019/11/19/cultivating-a-culture-of-partnership<p><em>This is cross-posted at the <a href="https://blog.stratasan.com/founders-perspective-cultivating-a-culture-of-partnership">Stratasan blog</a>.</em></p>
<h2 id="understanding-trust-and-transparency-are-three-keys-to-success">Understanding, Trust, and Transparency are Three Keys to Success</h2>
<p>I recently finished David McCullough’s “<a href="https://www.google.com/books/edition/The_Path_Between_the_Seas/uVaeIXMxJpQC?hl=en&gbpv=0">The Path Between The Seas</a>,” an overview of the planning and construction of the Panama Canal. The scale of the human effort put into this project is incomprehensible, but there is much to be learned from it about leadership, <a href="https://info.stratasan.com/hospitals-guide-to-achieving-organizational-alignment-email-course">organizational alignment</a>, and the importance of communication.</p>
<p>One way to think about alignment and communication is partnership. In the lead up to the construction of the canal, <a href="https://en.wikipedia.org/wiki/William_C._Gorgas">William Gorgas</a> was sent to Panama to deal with the problem of malaria and yellow fever. At the turn of the 20th century, many were still skeptical of the connection between these diseases and mosquitoes. When Gorgas tried to advocate for resources to reduce the mosquito population, he was constantly met with resistance and skepticism. Even though it was his job to reduce the tropical diseases that ravaged the canal worker population, he was unable to do so because his peers viewed him as a challenge to be overcome, not a partner in the construction of the canal.</p>
<p>The first <a href="http://www.pbs.org/wgbh/americanexperience/features/panama-canal-chief-engineers-panama-canal/">Chief Engineer</a> of the canal, John Wallace, saw Gorgas as an annoyance. As an engineer, he was concentrating on “making the dirt fly” and did not consider Gorgas’s medical background to be helpful with this effort. It wasn’t until <a href="https://www.britannica.com/biography/John-Frank-Stevens">John Stevens</a> became Chief Engineer that Gorgas was taken seriously. Stevens viewed Gorgas as a partner in a humanitarian effort. Once Gorgas was given the requested resources he was able to quickly eliminate cases of yellow fever.</p>
<p>Partnership—among individuals, departments, and other organizations—is pivotal to the success of any venture, whether it’s placing a new outpatient clinic or connecting the Pacific to the Atlantic over fifty miles of mosquito-infested jungle. So how do we cultivate partnerships?</p>
<h2 id="3-keys-to-cultivating-partnership">3 Keys to Cultivating Partnership</h2>
<h3 id="1-seek-to-understand">1. Seek to Understand</h3>
<p>As someone who straddles the line between the business and technical side, the partnership between the two is something I spend a lot of time thinking about. The best software engineers work hard to not only become masters of their craft, but also to understand the business problems that they are working to solve. Likewise, executives and managers benefit from understanding the technical side: software development methodologies, maintenance costs, and more.</p>
<p>Understanding incentives—how they differ and where they overlap—is pivotal in seeking understanding. Sales team members may be focused on revenue and retention, while tech team members may place more weight on lowering maintenance costs. Team members need to look for common ground: if an engineer is seeking to spend a significant amount of time paying down technical debt, it’s helpful to explain how not doing so might impact revenue in the near future.</p>
<p>Additionally, it’s always best to assume positive intent from others. If we start from the assumption that every team member is looking out for the health of the company, it’s much easier to find that common ground! That leads to the next way to promote a culture of partnership.</p>
<h3 id="2-first-trust">2. First, Trust</h3>
<p>It goes without saying that communication is important to a healthy partnership. However, writing or talking alone doesn’t get us far without trust. Much like reputation, psychological safety is hard to build and maintain and easy to destroy! An organization can spend years building up mutual trust, only to destroy it in weeks or even days.</p>
<p>Respectful listening and humility are both key to building and keeping trust. Both listening and humility require active participation—which is not the default stance most humans take when communicating. Notably, organizational leaders are in the terrifying position of being influential examples of this to the rest of the company. Their leadership is key to creating a culture based on trust.</p>
<h3 id="3-transparency">3. Transparency</h3>
<p>The only thing that’s worse than poor communication is no communication. Transparency means being honest when things aren’t going well, making information accessible and discoverable, and keeping the lines of communication open and clear. It sounds like such a simple thing, but much like trust, it’s difficult to build and easy to destroy.</p>
<p>The benefits of transparency extend beyond building trust: it’s even been proven to increase patient safety. The <a href="https://hbr.org/2018/11/building-a-culture-of-transparency-in-health-care">Harvard Business Review</a> explains that transparency, and ultimately, improved patient safety, happens when leaders create a “no-blame culture.” The article continues with “It is the responsibility of the leadership team to develop an atmosphere in which there is balanced accountability and continuous improvement.” When leaders are open and transparent, and when it’s demonstrated that transparency will be rewarded, then everyone will be more at ease adopting this approach.</p>
<h2 id="why-this-matters">Why This Matters</h2>
<p>Can you imagine if John Wallace had sat down with Gorgas and sought to understand why he asked for an incredible amount of resources to control the mosquito population in Panama? What if he had trusted that Gorgas as a partner, rather than seeing him as competition? Thousands of lives might have been saved.</p>
<p>Cultivating partnerships in an organization may not lead to such life-saving benefits, but it can definitely make or break a company. Much has been written about organizational alignment, but really they’re just fancy words for working together. It’s so much easier to work together if we’re headed in the same direction and we first seek to understand and trust!</p>
Commuting via eBike, 90 Days In2018-09-14T00:00:00+00:00http://dailytechnology.net//2018/09/14/ebike-commuting-90-days<p>Back in June I bought a <a href="https://www.electric-bike-kit.com/hill-topper.aspx">Hill Topper Electric Bike Kit</a> (Ranger, 36v motor). I mounted the kit on my Jamis Sport hybrid bicycle. It’s now September, about 90 days after I started using it, and I thought I would summarize the experience.</p>
<h3 id="prior-to-the-ebike">Prior to the eBike</h3>
<p>I have been commuting to work by bicycle for several years now. I did it with an old Wal-Mart bike, first, to prove that I could do it. After a few weeks I upgraded to a new Jamis Sport. It was a great bike for hauling kids around in a trailer, but quite heavy for some of the hills on my way to work.</p>
<p>After a few years, I upgraded to a used road bike. I love the light weight, but I often would take the bus to work to avoid getting sweaty, or maybe I just wasn’t feeling it that morning.</p>
<p>After seeing a couple of ebikes in the wild I decided to give one a try. My goals were:</p>
<ul>
<li>Shave a few minutes off my commute.</li>
<li>Smooth out some of the hills between my house and work.</li>
<li>Use the bike more often: not just for my commute to work but grocery runs, visits to the park, etc.</li>
</ul>
<h3 id="choices">Choices!</h3>
<p>There are so many eBikes and kits out there it’s hard to evaluate all of the options. I considered some of the lower end bikes, but most of the “cheap” ones are still in the $1200-1500 range and given that I already had two bikes that seemed a bit extravagant.</p>
<p>Kit prices vary wildly. The market appears to be flooded with a lot of cheap knock-offs and fly-by-night manufacturers. My priority was finding a company with a solid track record. If you run into issues with equipment, you want a company that will be around to answer questions. Furthermore, batteries don’t last forever so eventually they will need a replacement. Rather than guessing on compatibility, it’s nice to go with the company you originally bought the kit from.</p>
<p>I chose Hilltopper / Ukko because they seemed to match all that criteria. After making that choice, I was still confronted with a choice of 24 volt or 36 volt motor.</p>
<p>The 36 volt motor made more sense to me because it can top out around 15-20mph and provides a bit more oomph going up hills. I planned on also pulling a trailer with kids and/or groceries.</p>
<h3 id="kit-installation">Kit Installation</h3>
<p>Installing the kit was super easy. It’s basically just switching out the front tire and mounting the throttle and battery. Pretty much everything I needed, save the wrench, was in the box.</p>
<p>I did find that the cloth/canvas wrapping the battery eventually separated from the battery switch. It hasn’t caused any issues, other than the annoyance of occasionally trying to find the switch and see if I left the battery on (you typically want to turn it off when not in use).</p>
<h3 id="usage-patterns">Usage Patterns</h3>
<p>My ride to work is either 5 miles (short route, more traffic) or about 7 miles (longer route, less traffic).</p>
<p>The 36 volt motor tops out at a little over 20 mph on a flat road. This profile from one of my rides gives you a good idea of what speed I’m running. This is with me peddling a bit - I still want to get some decent exercise out of this! I imagine that you <em>could</em> ride without pedaling as much, but you’ll wear the battery out faster and, frankly, that’s boring. If you’re going to do that, you might as well go with an electric scooter.</p>
<p><img src="/img//2018-09-14-ebike-speed.png" alt="Riding hills" /></p>
<p>You <em>definitely</em> want to pedal along with the motor up hills. I haven’t tried it, but I’m pretty sure that on some of the steeper hills you might even come to a dead stop if you quit pedaling. You have to think of the motor (particularly the 24 volt one!) as a friend giving you a nice push. (Unfortunately, your friend also adds a bit of weight!)</p>
<p>I was a little concerned about having a front-wheel drive, but for the most part it hasn’t really been noticeable. If I’m accelerating and turning in loose gravel, I have to be careful because it tends to spin. Other than that, I rarely notice that the front wheel is doing most of the work.</p>
<h3 id="overall-take">Overall Take</h3>
<p>I have found that I met all my goals. I ride the work to bike much more often, and very rarely drive anymore. Even if I know I have meetings across town during the day, I can take the bike knowing that I won’t be sweaty for short rides. I also found that I’m more comfortable in traffic, which is a bigger deal than I anticipated. I can go from a stop to 20mph in much less time with the motor assisting my effort, and I can maintain a higher speed when going up hills.</p>
<p>I still commute with my road bike out on occasion, but much less often. I like how the ebike shaves about 5 minutes off my ride.</p>
<p>Really, the motor is just plain fun, too. I used to ride motorcycles, and while this is nothing like that, it still has a nice kick to it that makes for a pleasant ride.</p>
<p>Overall, I’m happy with my purchase. I drive less, and my commute is less stressful.</p>
<h2 id="the-ebike-future">The eBike Future</h2>
<p>There’s a bright future for ebikes - as more people discover them and they drop in price, we should start seeing more of them on the road. Commuters who might have been afraid to jump into the traffic fray on their own pedal power will be more comfortable given some electric assistance. I hope that this will encourage more people to trade four wheels for two, especially for those shorter, sub-10-mile commutes. In turn, that will increase the demand for more cycling infrastructure, which will (hopefully) morph into a virtuous circle, making it easier for even more people to cycle to work.</p>
<p><img src="/img/2018-09-14-bike.png" alt="My bike + kit" /></p>
<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/OipNWAuuUQE" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe>
Determining AWS Redshift Spectrum Spend2018-06-01T00:00:00+00:00http://dailytechnology.net//programming/2018/06/01/determining-aws-redshift-spectrum-spend<p>I’ve enjoyed using <a href="https://aws.amazon.com/redshift/spectrum/">AWS Redshift Spectrum</a> for quick-and-easy analysis of
fairly large-ish data sets using one of my favorite tools: SQL. However, as far as I could tell, there’s no way to see a
breakdown of Amazon Web Services Redshift Spectrum usage for your most recent billing cycle. You can, however, pull this
information from <a href="https://docs.aws.amazon.com/redshift/latest/dg/r_SVL_S3QUERY_SUMMARY.html"><code class="language-plaintext highlighter-rouge">SVL_S3QUERY_SUMMARY</code></a>.</p>
<p>Assuming that the pricing is still <a href="https://aws.amazon.com/redshift/pricing/#redshift-spectrum-pricing">$5 per terabyte</a> scanned, you can run this query.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>SELECT
SUBSTRING(querytxt, 1, 20) AS query_snippet,
s3.starttime,
s3_scanned_bytes,
(s3_scanned_bytes / (10^12) * 5) AS cost
FROM SVL_S3QUERY_SUMMARY s3 JOIN STL_QUERY q ON s3.query = q.query ORDER BY starttime DESC;
</code></pre></div></div>
<p>This will return a table something like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>+-----------------+----------------------------+--------------------+----------------+
| query_snippet | starttime | s3_scanned_bytes | cost |
|-----------------+----------------------------+--------------------+----------------|
| select foo, bar,| 2018-06-01 19:32:11.362227 | 209502504899 | 1.0475125245 |
| select foo, bar,| 2018-06-01 19:08:41.76035 | 209502504899 | 1.0475125245 |
| select foo, bar,| 2018-06-01 03:26:47.284144 | 347280195825 | 1.73640097912 |
| select foo, bar,| 2018-05-31 21:34:25.677985 | 209502504899 | 1.0475125245 |
| select foo, bar,| 2018-05-31 21:27:14.332106 | 209502504899 | 1.0475125245 |
+-----------------+----------------------------+--------------------+----------------+
</code></pre></div></div>
<p><em>Was this tip useful? Leave a comment and let me know!</em></p>
Using GNU Readline (Bash Shortcuts) in VS Code Terminal2018-05-30T00:00:00+00:00http://dailytechnology.net//windows/2018/05/30/using-gnu-readline-inside-vs-code<p>If you use VS Code and a Linux (bash/zsh/what-have-you) terminal, you may have noticed that some keyboard shortcuts (e.g., ctrl+p to cycle previous commands) are intercepted by VS Code. As a fairly <a href="http://dailytechnology.net/talk-cli-intro/#/">proficient user of shell keyboard shortcuts</a>, I found this to be pretty obnoxious. Here’s how you can fix it.</p>
<p>First, open your preferences file (hit ctrl+shift+p and type “User Settings”. You want to copy the default values for <code class="language-plaintext highlighter-rouge">terminal.integrated.commandsToSkipShell</code> and remove the following values:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> "workbench.action.quickOpen",
"workbench.action.quickOpenPreviousEditor",
"workbench.action.quickOpenView",
</code></pre></div></div>
<p>Once removed, you’ll find that ctrl+p should now properly rotate through your command-line history. Other shortcuts, like ctrl+k and ctrl+e should also work correctly (as they previously opened the workspace quick-open dialog).</p>
<p>There may be other commands that skip the shell, too. You can remove them from <code class="language-plaintext highlighter-rouge">terminal.integrated.commandsToSkipShell</code> as necessary.</p>
<p>Happy coding!</p>
<p><em>Was this tip useful? Leave a comment and let me know!</em></p>
Help! My Keyboard Layout Keeps Switching in Windows 102018-05-09T00:00:00+00:00http://dailytechnology.net//windows/2018/05/09/windows-10-keyboard-layout<p>I use the Colemak layout on Windows 10 version 1803 (April 2018). I’m aware that I can switch back to a Qwerty
layout by hitting Windows + Space, but occasionally my layout would switch anyway
and I knew I wasn’t hitting this shortcut. It turns out that there’s another keyboard
shortcut that does the same thing - either <em>Alt+Left Shift</em> or <em>Ctrl+Left Shift</em>. In
my case, it was the latter. Here’s how to fix this.</p>
<p>First, open settings. Choose <em>Time & Language</em> and then <em>Region and Language</em>.
Next, click <em>Advanced Keyboard Settings</em> below “Related Settings.” Then click
“Languag bar options” below “Switching input methods.” This opens a dialog box,
choose the “Advanced Key Settings” tab. You should see something like
this:</p>
<p><img src="/img/2018-05-09-windows-layout.png" alt="keyboard layout shortcuts" /></p>
<p>Click “Change Key Sequence” and then choose “Not assigned” for both “Switch
Keyboard Layout” and “Switch Input Language.”</p>
<p>Hit “Ok.”</p>
<p>Now you should be less prone to accidentally change your layout in the middle
of trying to get things done!</p>
Coaching/Mentoring Growing Developers2017-10-09T00:00:00+00:00http://dailytechnology.net//programming/2017/10/09/coaching-growing-developers<p>Let me hedge sharing this by saying: I am a terrible teacher. I lack patience, and when presented with the opportunity I will immediately jump to providing a solution rather than explaining the problem and working through it with a colleague.</p>
<p>That said, I do hope to get better at it. I have been reading Jo Boaler’s “What’s Math Got To Do With It?” and some of the questions that she recommends for encouraging students along seemed applicable to teaching software development, too.</p>
<blockquote>
<p>When I am called over to students who are stuck in math classes, I almost always start with “What do you think you should do?” Then, if I can persuade them to offer any ideas, I ask, “Why do you think that?” or “How did you get that?”</p>
</blockquote>
<p>This is a great way to kick off a conversation with a developer that may be stuck, too!</p>
<p>Boaler goes on:</p>
<blockquote>
<p>Some good prompts to use […] are:</p>
<ul>
<li>How did you think about the problem?</li>
<li>What was the first step?</li>
<li>What did you do next?</li>
<li>Why did you do it that way?</li>
<li>Can you think of a different way to do the problem?</li>
<li>How do the two ways relate?</li>
<li>What could you change about the problem to make it easier or simpler?</li>
</ul>
</blockquote>
<p>That last question is a great example of problem decomposition. When faced with a big programming problem (“I can’t get this program to run!”) how can we break it into smaller problems (“Let’s apply the <a href="https://en.wikipedia.org/wiki/Debugging#Techniques">wolf fence algorithm</a>!”).</p>
<p>The main thing to keep in mind is: be patient, encourage explanations (be the rubber duck!) and questions, and ask a lot of questions as the instructor.</p>
<p>I hope that I’ll get better at it. Adding these questions to my toolbox seems helpful.</p>
<p>Side note: if you have any interest in teaching math to kids, <a href="https://www.amazon.com/Whats-Math-Got-Transform-Mathematics/dp/0143128299/">“What’s Math Got To Do With It?”</a> is quite good. As usual with any good book that I find, I got it off my wife’s reading queue.</p>
Mapping NAD83 Projection Maps With GeoDjango2017-03-09T00:00:00+00:00http://dailytechnology.net//programming/2017/03/09/mapping-nad-83-data-with-geodjango<p>Placing shapes on a map is pretty straight-forward thanks to fantastic libraries like <a href="http://leafletjs.com/examples/quick-start/example-overlays.html">Leaflet.js</a>.
However, on one of my recent projects I needed to take maps that were using
NAD83 projections and translate them into usable <a href="http://geojson.org/">GeoJSON</a>.
Most mapping libaries used on the web use a more generic projection, WGS84, and
will not support drawing shapes using an alternative projection (like NAD83).</p>
<p>There are few tools better than
<a href="https://docs.djangoproject.com/en/1.10/ref/contrib/gis/">GeoDjango</a> and
<a href="http://postgis.net/">PostGIS</a> for building maps into web applications. I wanted
to use these tools to make the transformation.</p>
<p>If you search for “translating NAD83 to WGS84” you’re going to find all sorts of
intimidating information (like <a href="https://www.ngs.noaa.gov/CORS/Articles/WGS84NAD83.pdf">this whitepaper from
NOAA</a>). For a non-GIS
specialist like myself, this can be quite alarming. Fortunately, GeoDjango and
PostGIS have our back.</p>
<p>First, you’ll need to get GeoDjango and PostGIS up and running. That’s not in
the scope of this article, and there are plenty of wonderful tutorials on how to
do that so I will leave that as an exercise to the reader.</p>
<p>First, you need to get the SRID (also known as the EPSG) for the data you want
to map. You can get information about the file by inspecting it on the command
line with a tool like <code class="language-plaintext highlighter-rouge">ogrinfo</code> or using GeoDjango’s <code class="language-plaintext highlighter-rouge">DataSource</code> class.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ ogr2info path/to/filename.shp filename -so
</code></pre></div></div>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ ./manage.py shell
Python 3.5.2+ (default, Sep 22 2016, 12:18:14)
Type "copyright", "credits" or "license" for more information.
IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import os
In [2]: shp = os.path.abspath('/path/to/filename.shp')
In [3]: from django.contrib.gis.gdal import DataSource
In [4]: ds = DataSource(shp)
In [5]: layer = ds[0]
In [6]: layer.srs
Out[6]: <django.contrib.gis.gdal.srs.SpatialReference at 0x7f949428a8d0>
In [7]: print(layer.srs)
</code></pre></div></div>
<p>Either way you choose to do it, you will see something like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PROJCS["NAD_1983_UTM_Zone_16N",
GEOGCS["GCS_North_American_1983",
DATUM["North_American_Datum_1983",
SPHEROID["GRS_1980",6378137,298.257222101]],
PRIMEM["Greenwich",0],
UNIT["Degree",0.017453292519943295]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",0],
PARAMETER["central_meridian",-87],
PARAMETER["scale_factor",0.9996],
PARAMETER["false_easting",500000],
PARAMETER["false_northing",0],
UNIT["Meter",1]]
</code></pre></div></div>
<p>The part you want is that first line: <code class="language-plaintext highlighter-rouge">NAD_1983_UTM_Zone_16N</code>. Google that,
and you should be able to find a page like <a href="http://spatialreference.org/ref/epsg/nad83-utm-zone-16n/">this one on
spatialreference.org</a>.</p>
<p>Note that in this case, the SRID / EPSG is 26916. When you create a database
column in GeoDjango, you’ll either need to translate this to WGS84 or you’ll
need to indicate that the data is stored with <code class="language-plaintext highlighter-rouge">srid=26916</code>.</p>
<p>If you choose to store it in the original format, your model attribute will
look something like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class Foobar(models.Model):
geom = models.MultiPolygon(srid=26916)
name = models.CharField(max_length=32)
</code></pre></div></div>
<p>To convert <code class="language-plaintext highlighter-rouge">Foobar.geom</code> to GeoJSON, you can use the <a href="https://docs.djangoproject.com/en/1.10/ref/contrib/gis/serializers/">GeoDjango
<code class="language-plaintext highlighter-rouge">serializer</code></a>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>serialize('geojson', Foobar.objects.filter(name='Cookies'),
geometry_field='geom',
fields=('name',))
</code></pre></div></div>
<p>This will output a nice GeoJSON document that is projected in WGS84, useful for
showing on the web with a tool like Leaflet.js. Note that we did not need to
even provide the <code class="language-plaintext highlighter-rouge">srid</code> parameter here since it defaults to WGS84 (4326).</p>
Text Scaling Dell XPS 13 9350 and Ubuntu Gnome2016-08-15T00:00:00+00:00http://dailytechnology.net//hardware/2016/08/15/dell-xps-13-9350-and-ubuntu-gnome<p>I recently bought a Dell XPS 13 9350 and installed Ubuntu Gnome 16.04. I bought the version with the FHD (1920x1080) screen resolution. While that’s not exactly HiDPI, the text is rather small with the default installation.</p>
<p>If you, like me, have a difficult time reading the small text, you can bump it up with the Gnome Tweak Tool.</p>
<p>Open up the Tweak Tool and go to “Fonts.”</p>
<p>Adjust the “Scaling Factor” to something reasonable, like 1.25.</p>
<p>You may need to restart some applications to see the change.</p>
Commuter Cycling in East Nashville2016-02-10T00:00:00+00:00http://dailytechnology.net//2016/02/10/commuter-cycling-in-east-nashville<p>I want to talk about cycling for a moment. I live in East Nashville and I work
downtown. The neighborhood’s proximity to downtown is ideal for a daily bicycle
commute. My ride is approximately five miles, which can be accomplished in less
than half an hour. In spite of the nice bike lanes on my side of town, the
interstate creates a barrier that discourages commuters from cycling. Herein I
attempt to explain why.</p>
<p><img src="/img/2016-02-10-commuter-cycling-1-690px.png" alt="Map of routes into Downtown" /></p>
<p>The above illustrates all of the possible routes into downtown Nashville from
the east side. I did omit Crutcher Street, but it’s so close to Davidson Street
that I’m more or less lumping the two together.</p>
<p>Let’s take each of these options one by one, starting from the northernmost access.</p>
<h3 id="spring-street">Spring Street</h3>
<p>This street, theoretically, has a “bike lane.” Here’s a shot of it.</p>
<p><img src="/img/2016-02-10-commuter-cycling-spring-street.png" alt="Spring Street "bike lane"" /></p>
<p>I presume these chalked lines indicate that this is a “shared” road. Either that
or it’s the chalk outlines of where they found the bodies of cyclists fool-hardy
enough to take this route. We have multiple lanes of traffic, some of which
exit off of and enter onto Ellington Parkway. The only lane of choice for a
cyclist is the center lane where they are caught in merging traffic.
Furthermore, the road is in a curve so visibility is at a minimum. A vehicle
overtaking a cyclist would not see them until the last possible moment.</p>
<p>If you travel along this road headed into downtown, you’ll find yourself here.</p>
<p><img src="/img/2016-02-10-commuter-cycling-spring-street-2.png" alt="Spring Street "bike lane"" /></p>
<p>Here you will encounter cars coming off of Dickerson Pike, merging with
Interstate 65 North and South. You can ride in the right lane, but cars are
going to be annoyed at you for blocking their entrance and exit and yes they are
going to blow their horn and yell at you to get on the sidewalk. Beyond being
annoying, this is yet another horribly dangerous intersection that is not
suitable for commuting cyclists.</p>
<p>Here’s a close-up of the route where you can see all of the merging lanes and
general mayhem this route would have in store. It’s a veritable gauntlet.</p>
<p><img src="/img/2016-02-10-commuter-cycling-spring-street-route.png" alt="Spring Street Route Overview" /></p>
<p>I have driven this route for years now and I have yet to see a single cyclist on
Spring Street. This is, practically, not an option. To call it a bike route is
an insult to the English language. It is meaningless.</p>
<p>Let’s move on to the next route.</p>
<h3 id="main-street">Main Street</h3>
<p>This route, also, has a “bike lane.”</p>
<p><img src="/img/2016-02-10-commuter-cycling-main-street.png" alt="Main Street" /></p>
<p>Unfortunately, it is also afflicted with the problem of Interstate 65 ramps on
both sides. Again, the cyclist is forced into a center lane and into the wrath
of the motorized vehicles. You also have to contend with the fact that Main
Street funnels from two lanes to one lane when you cross 5th Avenue as well as
the Ellington Parkway off-ramp. All of this is quite intimidating, and even if
you obey the laws of the road, vehicles will be annoyed that you’re not over to
the side (in the merge lane).</p>
<p>The other side of the interstate isn’t much better. This is looking towards
downtown but you get a sense of what the street-scape is like in both directions.</p>
<p><img src="/img/2016-02-10-commuter-cycling-main-street-2.png" alt="Main Street" /></p>
<p>You’ve got five lanes of traffic. Typically during rush hour this is a very busy
street. Cars are entering and exiting the interstate. The middle lane on Main
Street south of the Interstate switches direction based on traffic. You have
five lanes of traffic all the way from the Interstate to downtown. Everything
about this road is set up to accommodate vehicles - there is very little
concession made to either pedestrians or cyclists.</p>
<p>Here’s an aerial shot of the whole mess.</p>
<p><img src="/img/2016-02-10-commuter-cycling-main-street-overview.png" alt="Main Street Aerial" /></p>
<p>This option is certainly better than Spring Street, but it’s still terrifying.
Let’s see what Woodland Street holds in store.</p>
<h3 id="woodland-street">Woodland Street</h3>
<p>This street has no designated bike route. However, at first glance it looks like
a really great route. Here’s a shot looking towards downtown.</p>
<p><img src="/img/2016-02-10-commuter-cycling-woodland-st.png" alt="Woodland Street" /></p>
<p>On the southbound side you have no exit or entrance ramps. Sure, there are two
lanes but otherwise this is pretty smooth sailing. No one is going to be mad at
you for riding in the right-hand lane. If you are coming south down Woodland
Street and you cross 5th Street, there’s a short length where cars behind you
must be careful about passing while there is only one lane. Alas, they are not
always careful, but one lane quickly becomes three, giving you plenty of room to
stay to the right.</p>
<p>This is actually my preferred route into downtown.</p>
<p>How about heading back home? Alas, we’re again faced with another entrance ramp.</p>
<p><img src="/img/2016-02-10-commuter-cycling-woodland-st-northbound.png" alt="Woodland Street Northbound" /></p>
<p>Not only is there an entrance lamp, but the far right hand lane is a turn lane.
This forces the cyclist into the second left-most lane. Again, we’re faced with
the wrate of the drivers encased in steel and glass who cannot understand why
this crazy man on a bicycle is slowing them down. They toot their horns, roll
down their windows, yell, swear, and generally act like imbeciles. I generally
do not take this route as I do not like to deal with angry drivers. It is, I
suspect, discouraging to anyone wishing to commute via bicycle.</p>
<p>You can, of course, choose to ride on the sidewalk but you will need to be very
careful about cars entering the interstate and turning onto Interstate Drive.
Furthermore, there have been several studies that have shown that <a href="http://www.bicyclinglife.com/Library/riskfactors.htm">riding on the
sidewalk is nearly twice as dangerous as riding in the
road</a>. Therefore, I do not
consider riding on the sidewalk here to be a viable option.</p>
<p>What are our other options, then? Let’s take a look at Shelby Avenue.</p>
<h3 id="shelby-avenue">Shelby Avenue</h3>
<p>This corridor into downtown, which includes the Korean Veterans Bridge, has
undergone a lot of changes in the past few years. There has been a bicycle lane
down Shelby for some time now, but it’s a little spotty in that it cuts in and
out and occasionally leaves you stranded on a fairly busy street.</p>
<p>Still, there <em>is</em> a bicycle lane and that’s certainly an improvement.</p>
<p>(Note the <a href="https://nashville.bcycle.com/">Bcycle-ist</a> running against traffic
patterns. This is something I see a lot of in Nashville - maybe the state
has not done a great job of educating folks about walking on the left, riding
on the right? It’s terribly awkward to try to safely pass these folks and deal
with traffic at the same time.)</p>
<p><img src="/img/2016-02-10-commuter-cycling-shelby-avenue.png" alt="Shelby Avenue" /></p>
<p>This route is a little more intimidating coming from the other direction. The
bike lane is there, but you have to really watch for folks moving to the left
lane (another Interstate on-ramp). They are technically supposed to yield to
cyclists, but they are not always great about doing so.</p>
<p>Still, there is a bicycle lane, so one would expect that motorists would start
to acclimate to seeing cyclists on the path.</p>
<p><img src="/img/2016-02-10-commuter-cycling-shelby-avenue-southbound.png" alt="Shelby Avenue Southbound" /></p>
<p>This route also experiences heavy traffic during rush hour (and heaven help you
if you decide venture this route on a game day). There are six lanes of traffic to contend
with, plus the on ramps and lane changes. It’s not something I have found to be
within my comfort level as a casual cyclist commuter. However, it’s not all
<em>that</em> bad. It does have proper lanes, at least.</p>
<p>If you’re coming from the north side of town, however (Dickerson Pike or
anywhere west of Gallatin Pike or Main Street) it is difficult to get to. You
will need to come down 5th Street (where there is no bicycle lane) or try to
cross on one of the smaller streets (e.g., 9th).</p>
<p>Last, let’s look at one option that has no interstate entrance ramps at all,
which makes it by far the least stressful option.</p>
<h3 id="davidson-street">Davidson Street</h3>
<p><img src="/img/2016-02-10-commuter-cycling-davidson-street.png" alt="Davidson Street" /></p>
<p>This street view is actually out of date. The bicycle lane you see above has
been replaced with a buffered split lane that runs along the south side of
Davidson Street. It’s quite nice (even if it does pose a problem for some
motorists who do not <a href="https://www.youtube.com/watch?v=QZJ9UwSN9jM&feature=youtu.be">look to their
right</a> when
entering Davidson Street). There are also no ramps onto I-65. It’s not beautiful
(it runs through a metals trash dump) but that can hardly be a valid complaint.</p>
<p>The only problem with this route is that if you need to travel in any direction
north of Nashville (anywhere west of Woodland street) it is rather
out-of-the-way. It adds two miles to my commute, but it’s such a lower level of
stress that I’m happy to add the additional time to get home.</p>
<h3 id="conclusion">Conclusion</h3>
<p>One of my favorite things about living in Nashville’s urban core is the ability
to take alternative forms of transportation to work. One of the reasons many people
choose to live in East Nashville is the proximity to the downtown area:
encouraging new residents to embrace alternatives will make the city a better
place to live by reducing traffic, pollution, and improve emotional and physical
well-being. Planners would do well to continue making efforts to make it more
attractive to casual cyclists. Here are some suggestions.</p>
<ul>
<li>Provide better routes from the north side of town, particularly the Dickerson
Pike corridor.</li>
<li>Make some concessions to pedestrians and cyclists on either Woodland or Main
Street in the form of buffered bike lanes, better sidewalk markings
(especially when crossing entrance and exit ramps) and minimizing lane shifts.</li>
<li>Educate cyclists that they are subject to the same rules as motorists.</li>
<li>Educate motoricsts that cyclists can take up a full lane and are to use the
road in generally the same manner as a vehicle.</li>
</ul>
<h3 id="other-resources">Other Resources</h3>
<p>If you cycle to work, make sure to read <a href="http://bicyclesafe.com/">BicycleSafe.com</a>.
You should be familiar with all of the common ways cyclists put themselves in
danger and how to avoid them. I try to go back and read it at least once a year.</p>
<h3 id="your-thoughts">Your thoughts?</h3>
<p>If you cycle in this part of Nashville, I would love to hear your opinion. What
do you feel can be done to improve the appeal of commuting to downtown via
bicycle to the casual cyclist?</p>
Beyond The Code: PyTennessee 2016 Keynote2016-02-07T00:00:00+00:00http://dailytechnology.net//development/2016/02/07/beyond-the-code-pytennessee-2016-keynote
<p>I was invited to give the closing keynote at PyTennessee 2016. Jason Myers,
Denise Myers, Will Golden, and the crew of volunteers continue to do an amazing
job putting on this conference. I’m grateful for all the work that Jason, in
particular, puts into this conference. I got front-row seats during his work on
the first conference, so I got to see how much work is involved in making a
regional conference happen. It’s a herculean effort.</p>
<div class="pull-right img text-center"><img src="/img/2016-keynote-250.jpg" class="img-circle" /><br /><small>Image credit: <a href="https://twitter.com/egdelwonk/status/696479086093459456">Will Golden</a></small></div>
<p>I owe a huge thanks to Jason Myers for giving me the opportunity to speak. I owe
another thank you to all of the folks who stuck around for my talk - it was, after
all, the final talk of a long weekend and it bumped up right against the Super
Bowl.</p>
<p>The video is available on <a href="https://www.youtube.com/watch?v=wU-dEeMNvR8&feature=youtu.be">YouTube</a>
(thanks to Scott Burns for recording) but reading is much easier than watching my
awkward presentation.</p>
<p>Here is the text of my talk. I did not use any slides, other than rotating
through some nice Impressionist art.</p>
<h3 id="we-are-all-creators">We Are All Creators</h3>
<p>In the spring of 1928, Walt Disney and Ub Iwerks were at the end of their
rope. Six months ago, they had created one of the most popular animated films in
the country - rivaling Felix the Cat and Koko the Clown. The problem, however,
was that the film’s main character did not belong to them.</p>
<p>The character, Oswalk the Lucky Rabbit, belonged to Universal Studios.</p>
<p>Since Universal owned the character, they could demand a lion’s share of the
profit. Walt made the trip to New York City to try to strike a bargain with
Universal - he wanted his company to enjoy a more significant share in the
Oswald’s success.</p>
<p>Universal, however, would not budge. If Walt walked away from the character,
they threatened to poach all his employees and continue marketing Oswald without
him.</p>
<p>On his trip back home, Walt came up with an idea for another character: Mickey
Mouse. In May of 1928, Disney and Iwerks released the cartoon they had
been working on in secret: Mickey Mouse’s “Plane Crazy.” It immediately surpassed
Oswald the Lucky Rabbit in popularity and the Disney company went on to become
what we know today.</p>
<p>What can we learn from this story?</p>
<p>Cesar Hidalgo of MIT (author of “Why Information Grows”) wrote about this story,
saying, “Oswald’s story teaches us that what is most valuable in an economy
is not what authors make, but their ability to make it. […] While legally,
Universal owned Oswald, technically they did not own the capacity to author
characters like Oswald. That capacity was embodied in the minds of Disney and
Iwerks, and this was the key factor that set the duo apart.”</p>
<p>That capacity - the capacity to create something unique - exists in all of us.
Whether you create with a brush and an easel, a drafting board, vim, or emacs,
you’re creating something out of nothing: castles in the sky.</p>
<p>The author of “The Mythical Man Book: Essays on Software Engineering” Fred
Brooks ruminated on the joys of our craft. The very first thing he mentions is
“the sheer joy of making things” (p. 8). There is a joy in creation.</p>
<p>Fred goes on to point out that beyond an innate joy in creation, there is a
joy in the creation of “things that are useful to other people.” We pour much
of our souls into our sky-castles and there’s nothing quite so demoralizing
as seeing our effort come to nothing as they crumble from neglect.</p>
<p>(As a parenthetical [hey, it’s a keynote, I’m allowed a parenthetical] I have one
PHP application that I wrote back in 2004 that is still happily tracking real
estate appraisals for a small firm in east Tennessee. I take no small
satisfaction in knowing that my simple little application has afforded a small
company to continue growing for over a decade now.)</p>
<h3 id="the-joys-and-woes-of-software-development">The Joys and Woes of Software Development</h3>
<p>But, as Fred Brooks points out in his classic book, not all is unicorns and
puppies. There are woes in our craft, too. Some of the woes that Dr Brooks
mentions:</p>
<ul>
<li>
<p>Computers do exactly what we tell them to do. This is true in that each
incantation must be uttered with utmost specificity and clarity. If we don’t
communicate our intentions correctly, we introduce all kinds of errors.</p>
</li>
<li>
<p>“Other people set our objectives, provide our resources, and furnish our
information.” We are at the mercy of those who are communicating requirements
to us, often filtered through third or even fourth parties.</p>
</li>
</ul>
<p>Dr Brooks wrote about these joys and woes over forty years ago, in 1975. What’s
remarkable to me is that any modern developer can affirm that these joys and woes
still apply to our day-to-day work.</p>
<p>What I also notice about these woes, in particular, is that they are completely
unrelated to your choice of language, platform, editing tool, operating system,
library, or any of the other things that we spend an inordinate of our time on.</p>
<p>In fact, I would argue that it’s easy to fall into a trap of spending <em>too</em> much
time on these things. We work in a trade that emphasizes self-directed learning
as an absolute necessity. It attracts those that are intellectually curious.
We’re always looking for ways to improve our ability to build better programs.
This is all well and good - our intellectual curiosity leads us to hone our
skills and build new and useful tools or make improvements to existing ones.</p>
<p>However, I think it’s easy to get stuck in the technical weeds. This is akin to
a painter spending a majority of his time debating which kind of paintbrush to
use, or which brand of canvas he should employ.</p>
<p>Resources:</p>
<ul>
<li><a href="https://hbr.org/2015/10/think-like-an-author-not-an-owner">Think Like an Author, Not an Owner</a></li>
</ul>
<h3 id="the-development-forest">The Development Forest</h3>
<p>Fred Brooks wrote another famous essay in 1986 titled “No Silver Bullet.” In it,
he makes the bold claim that “there is no single development, in either
technology or management technique, which by itself promises even one order of
magnitude of improvement in productivity, in reliability, in simplicity.”</p>
<p>Brooks puts the tasks of software development into two buckets, <em>essential</em> and
<em>accidental</em>.</p>
<ul>
<li>Our essential task is building the detailed conceptual structures that make
up the business rules. These are the same regardless of your chosen
architecture, language, or library.</li>
<li>Our accidental task is representing these abstract structures in a language
the machine can read and execute.</li>
</ul>
<p>Brooks argues that “the hard part of building software [is] the specification,
design, and testing of the conceptual construct, not the labor of representing
it and testing the fidelity of the representation.” That is to say, most of our
work occurs in the essential task of gathering requirements, building
specifications, etc. Less of our time is spent in the accidental task of
representing those specifications in machine-readable language.</p>
<p>Peopleware, a book that Brooks described as “one of his two favorite books on
software engineering” puts it this way:</p>
<blockquote>
<p>The business we are in is more sociological than technical, more dependent on
workers’ abilities to communicate with each other than to communicate with
machines. (p. 105)</p>
</blockquote>
<p>If this is indeed true, then this means that
improvements to existing languages and libraries are incremental at best. We
should be putting more of our efforts into building other skills that allow us
to become better at dealing with the <em>essential difficulties</em> of software
development.</p>
<p>This is not to say that spending time improving our ability to handle the
accidental difficulties is a waste. Indeed, as we are getting started in the
trade this is the best place to invest our time and effort. Our first hurdle is
to write instructions that a machine can understand. From there, we perhaps move
on to writing clearer, more maintainable code with fewer errors. (Books like
<a href="http://amzn.to/1lxvc8m">“Code Complete” by Steve McConnell</a> and <a href="http://amzn.to/1S5UeZ0">“Clean Code”
by Bob Martin</a> are excellent
treatises on improving these skills.) Eventually, we add additional languages to our
arsenal, giving us the added benefit of looking at the languages we already know
in new ways.</p>
<p>From there, however, things become unclear. What takes us from a developer that
writes great code to a developer that builds amazing products? To put it
simply, what happens when a junior developer becomes a senior developer?</p>
<p>Indeed, what makes a <em>good</em> developer? Is it writing clean, maintainable,
error-free code? Or, perhaps, could we take a step back from that and ask
ourselves: Are we building the right program? That is to say, do we even have
the right specifications?</p>
<p>Here we see ourselves debugging not the code, but the specifications. Our
ability to communicate with our stakeholders (our users, our team members)
becomes much more important.</p>
<p>Perhaps we can zoom out from simply debugging the specifications.</p>
<p>What can we as team members and, perhaps, team leaders encourage strong teams
where the whole is greater than the sum of its parts?</p>
<p>Or, zooming out even further: how do we encourage communities that foster
great developers, great teams to work with, and great companies to work at?</p>
<p>How do we encourage a trade that embraces a long-term outlook - a trade that
values ethical decisions over financial gain?</p>
<p>It may seem as if I’ve gone off the rails here, but I believe that this macro
vision has a profound effect on the micro level. The macro is the trade we work
in, the community in which we live, the employer we work for, and the team we
work together with. It sets the context for us to perform our work. It sets the
tone for our work and determines our ability to feel connected, valued, and
fulfilled as humans.</p>
<p>If you accept that this is true, then you must agree that it is important that
each of these layers work well in ways that push the roots of software development
(that is, the building of conceptual structures that we then translate into
machine-readable instructions) to succeed.</p>
<p>Imagine the accidental complexities of software development as the tips the
roots of a tree, deep in the soil. We could imagine the rest of the essential
difficulties as a tree.</p>
<p>A tree is insufficient. It’s bigger than that. It’s an entire forest.</p>
<p>What I want to talk about today is our place in that forest.</p>
<p>How can we <em>leave the place better than we found it</em>?</p>
<p>Additional resources:</p>
<ul>
<li><a href="http://www.connellybarnes.com/documents/language_productivity.pdf">Programming Language Productivity
Comparison</a></li>
<li><a href="http://worrydream.com/refs/Brooks-NoSilverBullet.pdf">No Silver Bullet, Fred
Brooks</a></li>
</ul>
<h3 id="mapping-the-forest">Mapping the Forest</h3>
<p>If we had a map of this forest, what would it look like?</p>
<ul>
<li>Ourselves</li>
<li>Teams</li>
<li>Employers</li>
<li>Community</li>
<li>World</li>
</ul>
<p>Each of these are building blocks - they are interdependent on one another.</p>
<h3 id="ourselves">Ourselves</h3>
<hr />
<p>The number of ways to grow as a software developer is equal to the number of
developers. As I was working on this talk I thought about talking about all
kinds of different things here: how to improve cognitive performance (learning
to learn), avoiding cognitive biases, becoming a craftsman… ultimately none of
these fit the talk because this is a talk about the social side of software
development: relationships.</p>
<p>You have daily interactions with your immediate team members. You talk to folks
in other departments about what they need from your team. Hopefully you also
find time to talk to your end-users.</p>
<p>All of that talking requires some serious communication skills. Before we can
get started on our sky castles, we need a conceptual model of what they’re going
to look like. It seems silly to say this because it’s listed as a requirement on
nearly every job out there, but good communication skills are pretty vital to
our trade. A software developer with no ability to communicate is going to have
a pretty hard time building anything useful for anyone other than herself.</p>
<p>How do we get better at communicating?</p>
<ul>
<li>Ask more questions and seek feedback. Good software developers are great
listeners. They are not only listening to the words that are being said, but
they are reading between the lines looking for clues as to what their users
really need. Even when talking to a product manager or designer, they listen
and ask clarifying questions about the specifications as they are being
delivered. When you first get started in a software development career, you
may not even know what questions to ask, but as you grow in the trade you find
that you can sense design gaps long before you start writing actual code.
This requires a nice dollop of humility.</li>
<li>Communicating well requires minimizing distractions. I confess that this is
a constant struggle with me. As a millennial (also known as a snake person)
I, like you, was born into an environment that is constantly
demanding context switches. It’s easy for us to lull ourselves into thinking
that we can pay attention to someone talking while we’re answering a text
message or scrolling through Twitter. The science is pretty solid on this:
when we spread our attention across several things, we do none of them well.
Furthermore, [multi-tasking damages our ability to
focus])http://news.stanford.edu/news/2009/august24/multitask-research-study-082409.html).
It is terrifying to think that this habit can destroy focus - arguably one of
the most important abilities of a software developer.
It also sends a signal to the person or persons that you’re talking to that
their time is not valuable.</li>
<li>I won’t spend a lot of time on this because I know Scott Burns did a great job
covering it in his talk, “Empathy as a Service.” The ability to listen is
great. To read between the lines, though, you often have to be able to place
yourself into the shoes of others. We had a client once who told us that that
he was having a terrible time updating his reports after we published some new
data. We asked to see what the trouble was, and he showed us how he would
print out Excel spreadsheets onto 14” long legal paper, whipped out a
calculator, and proceeded to write down a cumulative sum in one of the
columns. With a pen. It was easy for me to immediately wonder how this person
got out of bed in the morning, but one of my colleagues was gracious enough to
point out that this fellow had no experience with analysis. Due to some staff
cuts, he was a marketing person who had been shoe-horned into a position he
knew nothing about. Knowing that I was able to step away from my momentary
shock and dismay and take a moment to help this person do their job.</li>
</ul>
<p>Enough about ourselves. Let’s start talking about how we interact with the
forest around us.</p>
<h3 id="teams">Teams</h3>
<hr />
<p>Working with a team is one of the most complex things a software developer will
do. People are complicated. Every person is different, therefore every team is
different.</p>
<p>First, what does a team look like? A “team” is more than just a group of
people who have the fortune (or perhaps, misfortune) to work together. What
makes a group a team? Tom DeMarco and Timothy Lester call it “jell.”</p>
<blockquote>
<p>A jelled team is a group of people so strongly knit together that the whole is
greater than the sum of the parts. The production of such a team is greater
than that of the same people working in unjelled form. Just as important, the
enjoyment that people drive from their work is greater than what you’d expect
given the nature of the work itself.</p>
</blockquote>
<p>These kinds of teams rarely happen by happy chance. Does that mean that a
manager can build a jelled team? Au contraire. Management can create an
environment that allows a team to jell, but they can’t make it happen.
Furthermore, when it does happen, good management knows to get the heck out of
the way - their time is better spent moving obstacles out of the team’s way than
goading them on with sticks or carrots. Even <em>Peopleware</em>’s DeMarco and Lister
couldn’t come up with a list of six ways for managers to encourage great teams.
Instead, they ended up building a list of six things <em>not</em> to do.</p>
<p>Good teams are grown, not built. This follows nicely with our forest analogy,
doesn’t it? Just as you can’t force a seed to germinate, you can’t force a team
to jell. You can create an environment for it to happen. Furthermore, just as a
plant requires food and water to grow, a team requires ongoing effort to stay
jelled.</p>
<p>You may be thinking: I’m not a team lead - what can I do to help my team jell?
The ability to encourage good teams is not limited to just team leaders.</p>
<p>First, you can <em>embrace and welcome team diversity</em>. I’m not just encouraging
this for altruistic reasons. There is real research (detailed by
<a href="http://www.npr.org/2014/03/21/292225798/does-diversity-on-research-team-improve-quality-of-science">NPR</a>)
that suggests that diverse teams perform better. In this study they found that
research papers written by teams with diverse ethnic backgrounds received more
citations than papers written by authors with a similar ethnic background. As
Harvard economist Richard Freeman pointed out:</p>
<blockquote>
<p>Ethnic diversity is an indication of ideas’ diversity. People who are more
alike are likely to think more alike and one of the things that gives a kick
to science is that you get people with somewhat different views.</p>
</blockquote>
<p>If science can get a kick from a team with ethnically diverse backgrounds, is it
much of a stretch to assume that diverse software development teams would
receive the same benefit?</p>
<p>DeMarco and Lister also point out in <em>Peopleware</em> that a diverse team can send
a “clear signal that it’s okay not to be a clone, okay not to fit into the
corporate mold of a Uniform Plastic Person.”</p>
<p>Of course, to encourage diversity on your team it certainly helps to have some
diversity in our trade. We’ll talk about that later.</p>
<p>Next, teams should <em>discourage hero worship</em>. Perhaps you’ve been on the team
where there’s one person that everyone is in awe of. They sling code that no one
else understands, rescue projects that have fallen behind, and they enjoy a
level of autonomy that no one else on the team seems to get.</p>
<p>There are a couple of reasons to avoid having this mentality on your team.
One is that hero worship is a symptom of a fixed growth mindset.
A fixed growth mindset makes the assumption that a hero got that way because
they have innate talents that you can’t replicate. <a href="http://akaptur.com/blog/2015/10/10/effective-learning-strategies-for-programmers/">Allison
Kaptur</a>
did a great job of <a href="https://www.youtube.com/watch?v=Mcc6JEhDSpo">explaining this at Kiwi
Pycon</a>.
A fixed mindset has a detrimental effect on the ability of individuals to
learn and grow. A growth mindset, on the other hand, helps us remember that
skills are not innate, but can be grown with applied effort. This dovetails into
the oft-talked about “imposter syndrome.” As Allison puts it:</p>
<blockquote>
<p>Julie Pagano did a great talk at PyCon 2014 about impostor syndrome, and one
of her suggestions for a way to combat impostor syndrome was “kill your
heroes.” Don’t put other programmers on a pedestal, don’t say “that person is
so different from me.”</p>
</blockquote>
<p>The second issue is that it gives too much weight to the hero’s voice while
disenfranchising those who may have less experience. To illustrate this point,
I’m going to tell another story.</p>
<p>In 1977, the Citicorp Building (now known as Citigroup Center) was completed. It
features some startling architectural features, particularly at the base of the
building (<a href="https://en.wikipedia.org/wiki/Citigroup_Center#/media/File:Citigroup_center_from_ground.jpg">here’s a
photo</a>.
The entire 59-story building is elevated on stilts, but not just any stilts. Due
to a church in one corner of the property that could not be removed, the 9-story
stilts were placed in the center of each wall. This created some challenges in making
sure the building was structurally sound. The building is built with chevron
braces, which made it lighter than a normal skyscraper. This made it more
vulnerable to wind. To deal with that, the architect added a tuned mass damper</p>
<ul>
<li>one of the first ever built.</li>
</ul>
<p>A year after it’s completion, a student at Princeton University, Diane Hartley,
was reviewing the project as part of her civil engineering studies. She found
that the building architects had run all of their tests with wind coming
directly against the building faces (north, south, east, and west). What they
had not tested, however, was wind coming from one of the corners. By her
calculations, a strong wind against a corner would cause the building to lose
it’s structural integrity.</p>
<p>That is, the building would topple over.</p>
<p>Now, I don’t know about you but when I walk down the streets of Manhattan I
certainly don’t expect a building to tumble down in a strong wind.</p>
<p>Ms. Hartley did not believe that her calculations could be correct, so she
brought them to her professor. Her professor then passed them along to the
building architect, William LeMessurier. Fortunately, LeMessurier realized he
had made a glaring mistake and rushed to correct it.</p>
<p>Without informing the citizenry of New York, the problem was corrected by adding
some additional bracing to the building, and a major disaster was averted.</p>
<p>This entire story remained unknown until a <em>New Yorker</em> expose was published
in 1995. There’s also a great podcast about it from <a href="http://99percentinvisible.org/episode/structural-integrity/">99 Percent
Invisible</a>.</p>
<p>LeMessurier was a great architect but if it were not for the voice of a student
lives could have been lost. If the student had engaged in hero worship and had
not alerted the architect to the issue, the story could ended very differently.</p>
<p>That’s one kind of hero: the hero as the unquestioned expert. Another kind of
hero, the corporate hero, might look a little different. We’ve probably all seen
it - the one person who spent a whole weekend saving a project that went awry.
That kind of hero worship has an awful tendency to encourage bad
incentives. If someone has to spend all weekend knocking out an overdue project,
something went terribly wrong. That contribution should be appreciated, but
there needs to be some discussions what went wrong. Was the deadline
unrealistic? Were the specifications incomplete? Something else? If someone is a
corporate hero, that’s a great opportunity for some management introspection.
Why did we need a hero?</p>
<p>The third thing you can do to create an environment for your team to grow is to
<em>encourage a culture of craftsmanship</em>. It’s very difficult for a team to form
around the idea of “good enough” work. Indeed, poor quality work has a long-term
human cost in that team members become less and less enthusiastic about their
work with each slapdash project.</p>
<p>Craftsmanship doesn’t necessarily mean that we’re building cathedrals of
perfection. Gold plating can be just as harmful as leaving work unfinished.
We still live and work in a market society that has to ship projects
before our competitors. What it does mean, however, is balancing speed of
delivery with sound development practices.</p>
<p>One way to encourage this culture is to encourage <em>peer coaching</em>. If a team has
a culture of craftsmanship, team members feel it is acceptable to consult one
another for development advice. As DeMarco and Lister point out, the team lead
can’t know everything and often team members have a mix of complimentary skills.</p>
<p>For peer coaching to thrive, however, you need an environment where members don’t
feel inadequate for asking for help. A highly competitive atmosphere among team
members will discourage knowledge transfers that are important to the longevity
of the team.</p>
<p>There are a number of other ways to encourage the growth of good teams, but
these three will go a long way. If you’re interested in this subject, I highly
recommend Tom DeMarco and Timothy Lister’s book <em>Peopleware</em>.</p>
<p>Other resources:</p>
<ul>
<li><a href="https://www.thoughtworks.com/insights/blog/how-to-grow-effective-teams">How to Grow Effective Teams</a></li>
</ul>
<h3 id="employers">Employers</h3>
<hr />
<p>One afternoon when I was returning from a lunch-time run with my colleague Matt
George. Our new office was still under construction and the facade was covered
up with scaffolding. As we rounded the corner to the front of the building I was
looking up at he scaffolding in order to assess if we could get through the
front door or if we were going to have to circle around to the rear entrance. As
I was looking up, I took two steps into the wet concrete that had just been
poured for the new sidewalk.</p>
<p>My colleague, thankfully, was not looking up and managed to stop short of the
mess.</p>
<p>The general contractor was considerably annoyed by my misstep.</p>
<p>“What kind of an idiot,” he vociferously inquired, “doesn’t know what wet
concrete looks like?”</p>
<p>My co-founder, Jason Moore, said “Every person is an idiot outside of their
context.”</p>
<p>Now, leaving aside the fact that the contractor didn’t properly leave any kind
of visual signage that would have warned me about that wet concrete, Jason’s
response stuck with me.</p>
<p>One of the most grevious errors I’ve seen from developers in general is making
the assumption that non-developers are inferior beings. You’ve probably seen it
before. It’s so common that it has become a trope. You’ve probably seen the SNL
“Nick Burns, Computer Guy” sketch. Jimmy Fallon treats his idiot co-workers with
acute disdain, declaring “MOOOOOVE” as they try to explain the problem.</p>
<p>Most developers may not be this bad, but it’s easy to fall into this trap of
dismissing the input from non-techies. You’ll get a taste for what this feels
like if you travel to a non-English speaking country. Even if you have learned a
little of the language you struggle to communicate advanced concepts. Generous
locals will help you out, but the ungenerous ones may just assume you’re an
idiot.</p>
<p>This attitude of superiority is, of course, a cancer. It’s counter-productive to
successfully interpreting the needs of your non-technical colleagues. There have
been a few great talks on how important empathy is for software developers. That
applies here, too.</p>
<p>Company culture is admittedly difficult for one person to change. Indeed, even
those in a position of company leadership find it difficult to do so. Even
though it’s a challenge, it’s worth your consideration for two reasons:</p>
<ul>
<li>Your voice, along with others, can encourage change in the long term. At
worst, your change may be limited to your department or team, but if you
affect change, it’s worthwhile in that it increases your satisfaction with
your work as well as those around you.</li>
<li>You may come to find yourself in a leadership position to encourage change
(perhaps you already find yourself there).</li>
</ul>
<p>But what is company culture, exactly? If you’ve worked in industry for long
you’ve probably heard the term. Have you ever tried to define it? It’s pretty
difficult. Some interpret it to mean “company values” or the habits, attitudes,
and processes that exist within a company. That’s a pretty broad brush stroke.
It also discounts the fact that culture is something that is a constant state of
change. It changes in ways that are not always easy to observe, but companies
are, after all, made up of people. People are not known to behave in constant
ways. They are constantly changing, thus companies change.</p>
<p>So what is culture? <a href="https://hbr.org/2015/04/why-company-culture-is-a-misleading-term">One writer in Harvard Business
Review</a> put
it this way:</p>
<blockquote>
<p>I was interested in a paycheck and in order to get that paycheck, I had to
align my identity with the patterns of behavior and thought expected by those
who had power over me. (John Traphagan)</p>
</blockquote>
<p>That certainly puts it in stark terms. I think it’s pretty accurate, though.
Some companies make habits of trotting out platitudes of values for their
employees to consume like some sort of religious wafer:</p>
<ul>
<li>We believe in transparency and openness. (Possible interpretation: we read
your email and watch what websites you visit.)</li>
<li>We believe in working as hard as we play. (Possible translation: you’re going
to be working long hours, but we have a ping pong table!)</li>
</ul>
<p>Some companies have the best intent, but I am quite confident that any employee
at a company that is disingenuous about their values will catch on very quickly
as to what their <em>true</em> values are.</p>
<p>If we indeed place value on an idea, our actions will communicate that. If a
company really believes that “their employees are their greatest asset” then you
will see them make significant investments of time and money into their personal
development.</p>
<p>Culture, then, is how a company acts, not what a company says.</p>
<p>Earlier I mentioned that there is research that indicates that a diverse team
performs better than a homogeneous team. That is true, but to really enjoy the
benefits of diversity team members need to feel like they can speak up without
negative consequences.</p>
<p>That, fundamentally, means embracing a <em>culture of inclusivity</em>. That means not
only having a diverse set of backgrounds on your team, but also making sure that
their voices are heard and included when making important decisions.
Furthermore, research indicates that this additional step has a huge impact on
the ability of a company to respond to market opportunities that they may be
blind to.</p>
<p>A recent example of this was highlighted by Sylvia Hewlett of the “Center for
Talent and Innovation”:</p>
<blockquote>
<p>At Standard Chartered, for example, the insights of one Indian female banker
translated into innovation that boosted net sales at two branches by 127% and
75% within two years. Her insight? Local women—wage-earners, entrepreneurs,
and purse-string holders—were looking for a more welcoming, less condescending
experience when they went to deposit money, apply for credit, choose savings
and investment products, and ask for financial advice. Management endorsed her
proposal to make over the New Delhi and Kolkata branches into
all-women-staffed banks. Today, Standard Chartered is the go-to bank for women
in South Asia.</p>
</blockquote>
<p>Sylvia calls this “speak-up culture.” Many companies have a tendency to
disparage ideas from people who don’t look like them, and this costs them in the
long run. Collaboration suffers, innovation suffers, and risk-taking is
discouraged.</p>
<p>You may not be in a position of leadership, but there are still things you can
do to help encourage voices of others to be heard:</p>
<ul>
<li>Don’t drown them out. Give them an opportunity to speak.</li>
<li>Resist the temptation to “well, actually…”</li>
<li>Encourage management to listen (restate their views, if necessary, with proper
credit).</li>
</ul>
<p>One of the experiences that stuck out for me in 2015 was visiting a high school
in north Nashville to talk to a group of students interested in programming. I
was joined by a panel of other developers. Most of the class was made up of
Hispanic and African-American students. The panel was all white guys, save for
one white woman. (02/11 Edit: this is incorrect - I forgot, Marcus Whitney was on
the panel so it <em>mostly</em> white guys). One of the students, a sixteen year old girl,
was not afraid to name the elephant in the room:</p>
<p>“Looking at this panel full of white guys: do we even have a chance?”</p>
<p>The question made my heart sink. The only answer I could give was that our
industry desperately needs them. Our trade consists mostly of young, white males
and it shows in the kinds of applications that we develop and celebrate. Most of
it targets young, white males. If our businesses are to continue growing, we
need diverse voices like the ones I saw in that room to feel welcome and
participate.</p>
<p>If you <em>are</em> in a position of leadership, then you are obviously in a better
position to make cultural changes. I encourage you to take risks: real
leadership consists of more than just successful project or companies, but also successful
people. The projects you’re working on will come and go, but the members of your
team are going to to remember your management style even after they’ve parted ways.
If our trade is going to include the marginalized, it will start with you.</p>
<p>Now I want to transition into two areas that might get a little more heavy. I
say that because while I’ve got strong opinions about how we spend time on
ourselves and our teams, I have even stronger opinions on our interactions with
our community and our world. These go beyond just being a better developer. They
are where we become better humans. I argue that these things cannot be
approached independently - a terrible human is not a great software developer.
When you shuffle off that mortal coil how will you be remembered? Will it be
that absolutely fantastic open source library that you wrote? Or will it be how
you treated those you came into contact with?</p>
<p>The one person that really brought this reality home for me was Jim Weirich. MR.
Weirich was the creator of rake and a very active speaker in the Ruby community.
I got to see him speak at Ruby Hoedwon not long after I moved to Nashville. His
approachability was immediately noticeable. He was happy to answer questions,
even questions that others might dismiss as rudimentary. When he passed away in
February 2014 there was an outpouring of gratitude from those who had come into
contact with him. Sure, he was a great programmer and he had built some amazing
tools in the Ruby community. His lasting contribution, though, wasn’t his code.
It was his patience, kindness, and wisdom. That’s something to aspire to.</p>
<h3 id="community">Community</h3>
<hr />
<ul>
<li>“Community doesn’t just happen […]. It has to be made. The people who make
it are the unsung heroes of our work experience.”</li>
<li>We have a deep longing for community built into the human DNA.</li>
<li>Supportive communities welcome new members.</li>
<li>Communities contain embedded knowledge that accumulates over time.</li>
</ul>
<p>In the middle of the 19th century, Great Britian one of the world’s dominant
countries. It held sway over 10 million square miles. It was a world center
of science and progress.</p>
<p>In 1845, a Royal Navy officer, Captain Sir John Franklin, embarked to explore
the Arctic north of Canada. It was an ill-fated expedition. Later voyages found
the graves of the men and discovered that they had suffered from pnuemonia,
tuberculosis, lead poisoning, and scurvy. Furthermore, scratches on the bones
indicated that in their final days they had resorted to cannibalism.</p>
<p>In that same harsh climate in which they perished, however, you would have found
tribes of native Inuits who had survived for thousands of years.</p>
<p>The Inuits would have been labeled primitive by most 19th century standards. The
British certainly would have said as much. Why would they have survived in a
climate that a crew of capable English explorers could not?</p>
<p>The answer is that the Inuit community was embedded with the knowledge of how to
survive the cold. This kind of communal knowledge is both tacit and cumulative.</p>
<p>One of my favorite podcasters, Russ Roberts, points out that if the Inuits had
written books about how to survive in the wilderness, the English would still
have had a very difficult time adapting because much of the knowledge existed
in cultural practices and habits.</p>
<p>Furthermore, this knowledge was cumulative: The Inuit community had accumulated
survival techniques over generations: this is the kind of knowledge that would be
very difficult for one person to accumulate in a lifetime.</p>
<p>Community can happen anywhere. You have a community in your company, a community
in your neighborhood, perhaps a church community. You also have a community of
those who work locally in your trade. That community has embedded knowledge,
too. Knowledge about best practices, great places to work, what are “options”
and what do I do with them, how to run a conference… I have no idea how
how to put on a good conference. I’ve never done it before. This knowledge is,
however, embedded in the community in people like Jason Myers, Jacques Woodcock,
Will Golden, Ben Ramsey, and many others. (All we need to do, then, is harvest
their brains and… I’m just kidding!)</p>
<p>Much of what I covered in regards to teams and companies can be applied to
communities. If knowledge in a community is cumulative, then adding diversity to
our communities gives us a huge opportunity to grow that knowledge in new
directions. Every time we add a new member to our community, we add the
potential to grow the communities accumulated know-how. That’s one reason it’s
so important to make sure that our community is welcoming to newcomers.</p>
<p>How can we be welcoming to newcomers? One way is to make ourselves available for
knowledge sharing, or peer coaching. That’s fundamentally what user
groups are about - knowledge transfers and collaboration that encourage the
growth of an overall community. Developing the community’s professional skills
leads to companies setting up shop which encourages further growing of the
community’s professional skills which leads to… it’s a positive feedback
cycle.</p>
<p>Making it welcome to newcomers also means embracing diversity. I don’t just mean
demographic diversity, but a diversity of thought and background. That means
welcoming people who not only look like us, but maybe also don’t act like us or
think like us. That pushes our community to grow knowledge not only vertically
(writing “better” code) but also horizontally (what kinds of things are we
building, what needs are we serving).</p>
<p>Earlier I talked about how you can’t make a team jell. You can create an
environment for it, but you can’t force it. Communities, on the other hand, <em>are</em>
built. They don’t just happen. As DeMarco and Lister point out, “The people who
make [community] are the unsung heroes of our work experience.” This includes everyone
from the folks running the conference to those telling me when my time is up
and I need to leave the podium.</p>
<p>Indeed, what is more often missing in a community is <em>you</em>. Yes, you. You, like
me, have a multitude of other commitments that you must prioritize. Family,
work, etc. Or perhaps you find yourself imbued with a lack of confidence: how
can <em>I</em> contribute? I’m still swimming over here on the shallow end. In that
case, come talk one of the user groups about what you’re learning and how you’re
learning it. There are others who are right there with you and they’d like to hear
about your approach.</p>
<p>Opportunities to make an impact on the community level are often richer than the
opportunities found on a team or inside a
company. On a team or in a company your efforts may be stymied by management or
by the culture that is slow to change. On the community level, however, the
leaders tend to be those that show up regularly and give generously of their
time. There are endless opportunities to jump in and lend a hand - from speaking
at local user groups to volunteering at conferences to sponsoring.</p>
<h3 id="world">World</h3>
<hr />
<p>Most mature trades (legal profession, doctors and nurses, financial advisors)
have a well-defined code of ethics. In the mid 20th-century, in particular, many
trades and organizations created a code of ethics that still apply today.</p>
<p>Our trade has one, too. The Association for Computing Machinery first made a
stab at writing a code of ethics for programmers in 1972. It still actively
maintains a set of ethical principles that was also approved and adopted by the
IEEE (Institute of Electrical and Electronic Engineers).</p>
<p>What makes a code of ethics necessary? Often it is a response to market forces
(an attempt, for example, to avoid regulation by declaring, “Look! We’re self
regulating!”). I don’t think that was the case in the computing industry.
Regardless, it serves several purposes:</p>
<ul>
<li>It signals to the rest of the world that we can be trusted. Mark Frankel wrote
an oft-cited paper, “<a href="http://www.jstor.org/stable/25071878?seq=1#page_scan_tab_contents">Professional Code: Why, How, and with What
Impact?”</a>
and he put it this way:
“it will help persuade the public that professionals are deserving of it’s
confidence and respect, and of increased social and economic rewards.”</li>
<li>It communicates a list of ideals that we stand for. It is aspirational.</li>
<li>It “balances the risk of individuals with the risk of society” (see Jacob
Metcalf’s great report, “<a href="http://bdes.datasociety.net/council-output/ethics-codes-history-context-and-challenges/">Ethics Codes: History, Context, and
Challenges</a></li>
<li>It offers a method of self-retrospection: Am I, as a human, meeting these
standards that others in my profession have agreed are the standards?</li>
</ul>
<p>The first principle: “<em>Contribute to society and human well-being</em>.” That’s a
pretty heavy ask. This isn’t just a call to avoid hurting others (that
principle comes next), but to actively contribute to society and human well-being.</p>
<p>In a 1992 paper, the ACM put it this way, “[we have] an obligation to protect
fundamental human rights and to respect the diversity of all cultures. […]
computing professionals must attempt to ensure that the products of their
efforts will be used in socially responsible ways.”</p>
<p>If we, as a collective trade, abdicate our duty to protect fundamental human
rights (like the right to privacy) we will experience, as a trade, a collective
loss of public trust. If we lose that trust we will no longer enjoy the freedom to
build whatever we like, however we like. The trade could become stifled with
regulations and bureaucratic processes. It won’t happen all in one day or even one
year or even one decade. If you have worked in the healthcare or financial
industry, you probably have already had a taste of what this would look like in
the form of HIPAA and Sarbanes Oxley. Those regulations were a response to
developers and other computing professional that abdicated their responsibility
to the rest of humanity and betrayed their trust.</p>
<p>But this isn’t just about “not doing harm.” It goes beyond that. It’s
suspiciously close to what the popular HBO series “Silicon Valley” riffed on
during it’s first season: every startup had a variation on how they were going
to “make the world a better place.”</p>
<p>That may have become a platitude, but here we find it in original glory. As
professionals, it is up to each of use to evaluate our work and decide if it
meets the standard of “contributing to society and human well-being.”</p>
<p>The second principle may, at first glance, seem to be a subset of the first.
<em>Avoid harm to others.</em> The ACM elaborates on this by defining “harm” as “injury
or negative consequences.” They give examples like physical injury, loss of data
or property,” intentional or unintentional. This means we are to “carefully
consider potential impacts of […] decisions made during design.” This is
another opportunity for self-reflection during the specifications and design
phase.</p>
<p>There’s a scene in “Tron” (both the new and old versions) in which one of the
programs (Tron himself, I believe) says, “I fight for the users.” As the
architects of the sky castles, it’s our responsibility to fight for the users.
If necessary, we may even have to blow the whistle when we see harm being done.
It’s our professional responsibility.</p>
<p>Principle three: <em>Be honest and trustworthy.</em> Among other things, this applies to
our discussion earlier about an environment where team members don’t feel inadequate
when they ask for help. An honest developer isn’t ashamed to admit his or her
limitations. You can’t know everything, and pretending that you do is a
disservice to both your team and those that use your application.</p>
<p>Principle four: <em>Be fair and take action not to discriminate.</em> I like that this
principle doesn’t say “don’t discriminate.” This is a proactive declaration. We
all are guilty of discriminating in different ways (it often happens without any
self-awareness). This is urging us to take action to be inclusive, to take
action when we observe discrimination, to fight for equality when we find the
opportunity before us.</p>
<p><em>Give proper credit for intellectual property.</em> Those licenses you see on
software packages are important. We’re obligated to respect the restrictions
placed on software by others, even if we don’t like them.</p>
<p><em>Respect the privacy of others.</em> This one is particularly fascinating to me
because I’ve been reading Bruce Scheier’s “Data and Goliath.” As Scheier puts
it “surveillance is the business model of the internet.” Content is only
valuable in that it attracts eyeballs that can be sold to advertisers. The more
advertisers know about those eyeballs, the better. Google knows everything there
is to know about me - everything I say (thanks, voice control!), how many steps
I take, the places I visit. The potential for abuse of this information is
significant. I certainly hope that the developers at Google have read the ACM’s
code of ethics and decided that respecting the privacy of others trumps padding
their wallets.</p>
<p>The ACM elaborates on this principle by saying, “This imperative
implies that only the necessary amount of personal information be collected in a
system, that retention and disposal periods for that information be clearly
defined and enforced, and that personal information gathered for a specific
purpose not be used for other purposes without consent of the individual(s).”</p>
<p>This is not something that most companies today are doing. Often we can
only guess about what information is being collected, how long it will be
retained, or how it will be used. Again, this could end badly if we abdicate
our professional responsibility of respecting the privacy of our users.</p>
<p>That ties into the last principle: <em>Honor confidentiality.</em> As developers we often
have access to a pile of information that most people don’t. Betraying that trust
cheapens our trade.</p>
<h3 id="conclusion">Conclusion</h3>
<hr />
<p>One of my favorite feminist writers, Dorothy Sayers, wrote the following in her
essay “Why Work?”:</p>
<blockquote>
<p>We should ask of an enterprise, not “will it pay?” but “is it good?”; of a
man, not “what does he make?” but “what is his work worth?”; of goods, not
“Can we induce people to buy them?” but “are they useful things well
made?”; of employment, not “how much a week?” but “will it exercise my
faculties to the utmost?”</p>
</blockquote>
<p>A tree without sturdy roots will not make a healthy forest. I acknowledge that
we need to spend a significant portion of our time focussing on our ability to
develop better software. Some of that time will be spent making sure that we are
capable of being the best developer we can be.</p>
<p>Beyond that, though, is so much more. Imagine you’re telling your grandchildren
stories about your adventures as a software developer in the first half of the
twenty-first century. Will you regal them with stories of the clever one-liners
you wrote in Rust? Perhaps you’ll tell them about the time you saved a company
by making an algorithm faster, or cleaner, or more maintainable (perhaps all
three!). Maybe you’ll elaborate on how you took the test suite from 50% code
coverage to 90% code coverage.</p>
<p>Perhaps you would tell these stories, and perhaps your grandchildren would be
duly impressed. I am willing to bet, however, that it will be stories about your
<em>relationships</em> in the trade that stand the test of time.</p>
<p>Would not you rather tell them about how you, with your privileged status as a
white male with straight teeth, worked to with your team members to welcome the
marginalized into your company, giving them a voice to contribute their ideas?</p>
<p>Wouldn’t you rather tell them about how you joined that team and worked closely
alongside an amazing group of people to build something that contributed to the
betterment of humanity?</p>
<p>Wouldn’t you rather tell them about the community that you were part of, how you
coached the newcomers and challenged them to become better developers,
contributors, and humans?</p>
<p>I don’t know about you, but that’s the legacy that I’d like to leave behind.</p>
<p>Let’s leave that forest thriving.</p>
<p>Thanks for your time.</p>
Care and Feeding of Your Brain2016-01-28T00:00:00+00:00http://dailytechnology.net//development/2016/01/28/care-and-feeding-of-your-brain<p>The number of ways to grow as a software developer is equal to the number of
developers. One commonality is that the learning never stops. We will always
have new languages, libraries: abstractions on top of abstractions on top of
abstractions.</p>
<p>This is just the nature of the trade. It’s a factor what makes it challenging to
newcomers. It also means that it’s important to learn how to learn. You want to
be a well-rounded person with interests outside of software development, so it
follows that you would want to get the best return on your time spent covering
new material.</p>
<p>One of the ways to do this is to spend some time learning about how the brain
works. A great resource that I’ve relied on is Dr. John Medina’s <em>Brain Rules</em>. There
are not a few books out there on brain science, but one of the reasons that I
like Dr. Medina is that he only relies on peer-reviewed, replicated studies.</p>
<p>It should go without saying that the brain is immensely complicated. As Medina puts
it, “if we ever fully understood how the human brain knew how to pick up a glass
of water, it would represent a major achievement [in science].”</p>
<p>Medina breaks down what we know about the brain into 12 rules. I’m not going to
cover them all - I could never do them all justice. (Although I do highly recommend the
book if you ever get a chance to read it.) I’m just going to highlight a couple of
them that I found to be particularly relevant to those of us in the software
development trade. If you’d like to read about all of them, check out
<a href="http://brainrules.net">brainrules.net</a>.</p>
<p><em>Rule 1: Exercise Boosts Brain Power</em></p>
<p>Medina’s first brain rule is <em>Exercise boosts brain power.</em> Our bodies were
meant to move. We hunted, farmed, and ran from predators since the beginning of
time. As software developers we hunt bugs, farm unit tests, and the only thing I
run from is our CEO when he asks for next year’s conference sponsorship budget.</p>
<blockquote>
<p>“There is no greater anti-brain environment than the classroom and cubicle.”
<em>(Medina)</em></p>
</blockquote>
<p>Fundamentally, exercise improves cognition by improving blood flow in our
brains. Blood flow is connected to gathering oxygen. Getting that oxygen to our
brain is pretty darned important. If you don’t believe that, just try holding
your breath for the next five minutes.</p>
<p>Imagine if you had to do your job using a 2400 baud modem. It would really
hamper your ability to read Stack Overflow, wouldn’t it? You’d prefer to have a
nice symmetrical fiber connection, preferably a gigabit one.</p>
<p>I grew up in a rural community in east Tennessee that even now still does not
have cable or DSL available. For many year, the best option was Hughesnet
Satellite (because “what else are you going to do?”). At one point, my dad
declared that he had figured out how to make his downloads faster. I was
immediately nervous that he had downloaded some kind of malware advertising
better compression.</p>
<p>“How did you make it faster?” I asked.</p>
<p>“Well, I get myself a nice glass of scotch and sit down at the computer. I type
in the web page, and then I sip on glass of scotch and take a nap. When I wake up,
the web page has finished loading!”</p>
<p>(Note: I do not recommend this method of boosting internet speeds. Particularly
at work.)</p>
<p>A faster connection doesn’t necessarily mean you can access new content, it just means
your <em>access</em> to it is much easier. Similarly, exercise “stimulates the blood
vessels to [improve flow].” It allows that blood, and the important oxygen and
food that it delivers, to “penetrate deeper and deeper into the tissues of the
body” [and brain].</p>
<p>Medina covers several other ways that exercise improves cognitive ability, too.</p>
<blockquote>
<p>“The benefits of exercise seem nearly endless because its impact is
systemwide, affecting most physiological systems.” <em>(Medina)</em></p>
</blockquote>
<p>Ultimately, I run because I know it’s good for my brain and my body.
Penultimately, I work with a crew of really smart folks and if I ever hope to be
able to keep up with them I have to keep running to stay sharp.</p>
<blockquote>
<p>“Exercisers outperform couch potatoes in tests that measure long-term memory,
reasoning, attention, and problem solving skill.” <em>(Medina)</em></p>
</blockquote>
<p>According to Medina, the gold standard for exercise ROI is 30 minutes of aerobic
exercise two or three times a week. That doesn’t seem like a big ask to me.</p>
<p><em>Rule 2: Sleep well, think well.</em></p>
<p>This one should seem self-obvious - we all know that it’s important to get a
good night’s sleep. It’s almost become something of a platitude. However, the
science most certainly backs it up. If you want to learn well or perform well
rest is very important.</p>
<p>I say “rest”, but what’s going on in your brain at night really isn’t
very restful. During REM phases your brain is quite active. There are valleys
and peaks of activity all through the night. Those peaks, the REM cycles, are
doing something important to your cognitive ability. One study has shown that
if you are teaching rates to navigate a maze and you interrupt their sleep
during the REM cycle they do not perform as well. It hampers their cognitive ability.</p>
<p>Medina spends a lot of time going over the battle between the circadian arousal
system (the process that wakes you up) and the homeostatic sleep drive (the
process that tells your body it needs to rest). It is important to note that
most brains reach a stalemate in this battle in the mid-afternoon.
This leads to the afternoon dip that most of us experienced.
There are studies that have shown that a short twenty minute nap during the
afternoon measurably improves cognitive ability in the remainder of
the day. Medina points out that if you’re a speaker, avoid the mid-afternoon. It’s
the most difficult time of the day for most folks to pay attention.</p>
<p>Those are just two of the twelve brain rules that Medina covers in his book.
Some of the other rules cover stress, attention (you can’t multi-task as well as
you think you can), and the importance of exploration. If you’re interested in
this subject, check our his book. You can also get a nice outline of all of his
rules at <a href="http://brainrules.net">brainrules.net</a>.</p>
Software Development Is More Than Coding2015-10-27T00:00:00+00:00http://dailytechnology.net//development/2015/10/27/software-is-more-than-code<p>“What language should I focus my efforts on?” It’s one of the most common
questions I hear when talking to aspiring software developers. When the question
is posed they are disappointed that most senior developers refuse to give them
a clear answer. Basically, the language choice itself is relatively minor
compared to all of the other things you need to focus on to grow as a software
developer.</p>
<p>(To be fair, the question really boils down to “What language has the most
demand (and therefore makes it easier for me to find a job)?” In that case, the
answer is best obtained via a <a href="https://www.google.com/search?q=most+popular+software+language&ie=utf-8&oe=utf-8">quick
search</a>.)</p>
<p>Frederick P. Brooks, Jr.’s classic essay <a href="http://worrydream.com/refs/Brooks-NoSilverBullet.pdf"><em>No Silver Bullet — Essence and Accident in Software
Engineering</em></a> is one of my
favorite essays found in one of my favorite development life cycle-related books,
<em>The Mythical Man Month</em>. His fundamental argument is as follows:</p>
<blockquote>
<p>There is no single development, in either technology or management
technique, which by itself promises even one order-of-magnitude
improvement within a decade in productivity, in reliability, in simplicity.</p>
</blockquote>
<p><em>Technology</em>, in the above quote, includes programming languages. There really
is no “best” language to learn. Some are easier to learn then others; some are
better suited to particular tasks; others suit programming styles in ways that
please particular developers.</p>
<p>Dr. Brooks goes on to say:</p>
<blockquote>
<p>I believe the hard part of building software to be the specification, design,
and testing of this conceptual construct, not the labor of representing it and
testing the fidelity of the representation. We still make syntax errors, to be
sure; but they are fuzz compared to the conceptual errors in most systems.</p>
</blockquote>
<p>To put it another way: writing the code is the easy part. Specification, design,
and testing are the real meat of software development. These are the tasks that
are both difficult to teach and difficult to learn. Teaching to write code, in
comparison, is relatively simple.</p>
<p>Dr. Brooks goes on to cover some of the hopes for the silver bullet. In one of them
he even alludes to writing comprehensive tests (he calls it <em>program verification</em>).</p>
<blockquote>
<p>More seriously, even perfect program verification can only establish that a
program meets its specification. The hardest part of the software task is
arriving at a complete and consistent specification, and <em>much of the essence
of building a program is in fact the debugging of the specification</em>.</p>
</blockquote>
<p>(Emphasis added.) Again, Dr. Brooks concludes that the “essence” of software development
is in refining the specification.</p>
<p>What is the takeaway here for an aspiring software developer?</p>
<p>In my humble opinion, the best developers are <em>good communicators</em>. They may not
know how to write a <a href="https://en.wikipedia.org/wiki/Doubly_linked_list">doubly linked
list</a> from scratch but they
can take a incomplete specification and:</p>
<ul>
<li>Conceptualize how it will work</li>
<li>Refine the specification to make it more… specific.</li>
<li>Identify traps that may cause errors in communication</li>
</ul>
<p>Don’t spend too much time deciding what language you should learn. The real meat
of software development is problem-solving. Find a problem. Solve it. Doing so
might even involve writing some code.</p>
<p>If you haven’t read it, Dr. Brook’s essay is online in it’s entirety. Even though
it was written in 1986, it’s still very much worth a read. At 16 pages, it won’t
even demand much of your time. Find it
<a href="http://worrydream.com/refs/Brooks-NoSilverBullet.pdf">here</a>.</p>
2015 PyCon: Day 12015-04-10T00:00:00+00:00http://dailytechnology.net//python/2015/04/10/2015-pycon-day-1<p>I have been fortunate enough to attend four PyCon conferences. I find that it
helps me to maximize my experience if I take notes and summarize them each day.
Perhaps I will later roll this into a post that summarizes the entire
conference.</p>
<p>Notes from my day:</p>
<ul>
<li>PyCon puts all of their talks into <a href="https://play.google.com/store/apps/details?id=com.guidebook.android">GuideBook</a>. Use it! It’s a great way to manage a schedule!</li>
<li>If you can avoid it, <em>don’t</em> open your laptop during a talk! It’s terribly distracting and data has shown that <a href="http://www.scientificamerican.com/article/a-learning-secret-don-t-take-notes-with-a-laptop/">you retain more information if you take notes by hand</a>.
<ul>
<li>Sure, monitoring IRC and/or Twitter may be great, but know that it comes with a cost.</li>
</ul>
</li>
</ul>
<p>My notes for each talk:</p>
<h3 id="opening-statement-julia-evans">Opening Statement: <a href="https://twitter.com/b0rk">Julia Evans</a></h3>
<p>Wow, is Julia enthusiastic or not? She gave a whirlwind overview of PyCon.
Highlights included the usual Code of Conduct, their efforts to increase
diversity of speakers and attendees, and the $200,000 in donated financial aid
(aiding 291 attendees, or more than 10% of the total).</p>
<h3 id="keynote-catherine-bracy">Keynote: <a href="https://twitter.com/cbracy">Catherine Bracy</a></h3>
<p>Catherine is the Director of Community Organizing at Code for America. She
doesn’t have a technical background, from what I gathered, but she is quite
passionate about getting the populace engaged in goverment. I’m a bit of a
pessimist on this front, especially after reading <a href="http://en.wikipedia.org/wiki/The_Myth_of_the_Rational_Voter">The Myth of the Rational
Voter</a>, but I
appreciate her enthusiasm and wish her organization the best of luck.</p>
<p>Her talk was very eloquent. She pointed out many of the glaring inefficiencies
in government-led IT (the most high-profile of which was the Healthcare.gov
crash as the Affordable Care Act went into effect). Other examples included high
school students in Los Angeles sitting in an auditorium while their class
scheduling system was sorted out and Floridians being unable to claim unemployment
benefits because of issues with their online-only tool.</p>
<h3 id="building-secure-systems-lvh">Building Secure Systems: <a href="https://twitter.com/lvh">lvh</a></h3>
<p>I am an avid reader of <a href="https://www.schneier.com/">Bruce Schneier</a> ever since
attending <a href="http://www.thelasthope.org/">the Last HOPE</a> conference (associated
with the venerable 2600 hacker quarterly), so I appreciated lvh’s talk. He
highlighted the fact that typically our job as developers is to <em>make it work</em>,
not to make it fail; the latter is how security operations are tested. He
mentioned that new features will always be more exciting and interesting than
security in the form of bugs. As he pointed out, this is a rational
self-interested decision many companies make: if you look at market cap, it
doesn’t really shift much after a huge security disclosure. He did not say this,
but ultimately, consumers will have to become more security-sensitive for this
lack of priority to ever change.</p>
<p>lvh mentioned that most computer science courses include the <a href="http://en.wikipedia.org/wiki/Liskov_substitution_principle">Liskov
substitution principle</a>
as part of their cirriculum, but they do not include the <a href="http://en.wikipedia.org/wiki/Principle_of_least_privilege">principle of least
authority</a> (or
privilege), which ultimately might be a more useful topic.</p>
<p>He also mentioned a free resource he put together,
<a href="https://www.crypto101.io/">crypto101.io</a>:</p>
<blockquote>
<p>Crypto 101 is an introductory course on cryptography, freely available for
programmers of all ages and skill levels.</p>
</blockquote>
<p>I really enjoyed his talk, although it covered security on a sufficiently high
enough level that I have heard most of it before.</p>
<h3 id="introduction-to-https---a-comedy-of-errors-ashwini-oruganti">Introduction to HTTPS - A Comedy of Errors: <a href="https://twitter.com/_ashfall_">Ashwini Oruganti</a></h3>
<p>Ashwini had given this talk at
<a href="https://www.pytennessee.org/schedule/presentation/84/">PyTennessee</a> and I heard
good things about it, so I was happy to catch it at PyCon. It is quite obvious
that this contributor to a pure Python TLS implementation is quite familiar with
how HTTPS works.</p>
<p>She first mentioned out that
<a href="https://docs.python.org/2/library/urllib.html"><code class="language-plaintext highlighter-rouge">urllib</code></a> is fundamentally
flawed in that it does not do any kind of server certificate validation.
Any fool with <a href="http://ettercap.github.io/ettercap/">ettercap</a> and
<a href="https://mitmproxy.org/">mitmproxy</a> can give you a bad time.
Fortunately, there are alternative tools at the Pythonista’s disposal (e.g.,
Python 3 and the <a href="http://docs.python-requests.org/en/latest/"><code class="language-plaintext highlighter-rouge">requests</code></a>
library).</p>
<p>Furthermore, says Ashwini: SSL is busted; use TLS.</p>
<p>She pointed to a few nice resources, not least of which was <a href="https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/">Hynek’s guide to
configuring
TLS</a>, which
has has been kept updated. She also mentioned the book <a href="http://www.amazon.com/gp/product/1593273886/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=1593273886&linkCode=as2&tag=rea036-20&linkId=ZRZVXBNZ2WBTZUVI">The Tangled
Web: A Guide to Securing Modern Web Applications</a>.</p>
<p>The big takeaway from this talk: test your clients against servers with bad
certificates. They should fail! If they don’t, fix that.</p>
<h3 id="a-dive-into-tls-benjamin-peterson">A Dive into TLS: <a href="https://twitter.com/gutworth">Benjamin Peterson</a></h3>
<p>This talk was retitled “The (Less) Sorry State of SSL in Python” and it did
indeed turn out to be an overview of the how things have changed in the Python
library since <a href="https://hynek.me/talks/tls/">Hynek’s
talk titled ‘The Sorry State of SSL’</a>.</p>
<p>As I don’t really use Python to consume TLS, this talk was not quite as useful
to me. Peterson, however, is a very engaging speaker.</p>
<h3 id="distributed-systems-101-lvh">Distributed Systems 101: lvh</h3>
<p>This talk was so over my head I had a really hard time keeping up. Distributed
systems is not something I spend a lot of brain power one (if I need something
distributed, I tend to rely on services to fill that gap). lvh went into the
talk saying that he wanted to impress everyone with how hard distributed systems
are, and I suppose he did just that.</p>
<p>My biggest takeaway was that there are three things you want from distributed
systems. If you draw them in circles and imagine a Venn diagram, you cannot get
all three. You can only pick 2.</p>
<ul>
<li>Consistency</li>
<li>Availability</li>
<li>Partition tolerance</li>
</ul>
<p>Basically, the last one, partition tolerance, is not really optional. So most
systems concentrate on the gradient between consistency and availability.</p>
<h3 id="smart-services--smart-clients---how-microservices-change-the-way-you-build-and-deploy-code-frank-statton">Smart Services & Smart Clients - How Microservices Change the Way You Build and Deploy Code: <a href="https://twitter.com/frankrstratton">Frank Statton</a></h3>
<p>Frank is the CTO and Runscope, and he took a nice in-depth look at how Runscope
manages their infrastructure in ways that minimize tight coupling.</p>
<p>I had a splitting headache during this talk so unfortunately I had to make an
exit to hunt down some pain medication. Alas, it appeared to be a good talk.
I’ll have to catch the video later.</p>
<h3 id="stop-sucking-me-into-your-drama-a-personal-appeal-for-loose-coupling-augie-fackler-nathaniel-manista">Stop Sucking Me Into Your Drama: A Personal Appeal for Loose Coupling: Augie Fackler, Nathaniel Manista</h3>
<p>I really enjoyed how these two tag-teamed their talk. It kept the topic fresh and
entertaining as they would bounce back and forth.</p>
<p>This topic is similar to what you would read in Bob Martin’s Clean Code or Steve
McConnell’s Code Complete.</p>
<h3 id="coreos-dan-callahan">CoreOS: <a href="https://twitter.com/callahad">Dan Callahan</a></h3>
<p>I’ve used the lightweight Linux distro CoreOS for managing Docker on Mac OS X,
but I had no idea that the distro had so many things going on.</p>
<p>First Dan went over the basics of why we like containers. After that, he went
over:</p>
<ul>
<li>FastPatch for opportunistic, recoverable upgrades.</li>
<li>Docker for containerization</li>
<li>etcd for consensus
<ul>
<li>If updates are automatically applied, how do I make sure my system is stable? etcd is the answer!</li>
<li>But wait, if etcd is also running on CoreOS, how do I make sure my system is stable? The answer is more etcd! Dan recommended 5 etcd nodes.</li>
<li>Maor virtual machines!</li>
</ul>
</li>
<li>Fleet for scheduling.</li>
</ul>
<p>If I had one word to describe CoreOS it would be “clever.” Of course, sometimes
clever can get you into trouble.</p>
<h3 id="board-games">Board Games</h3>
<p>After dinner I joined a group of Pythonistas to play some board games generously
provided by <a href="http://www.randolph.ca/?lang=en">Randolph’s Pub</a>. This pub has
thousands of games and you can go there to drink beer and play. How fantastic is
that? May have to visit them before I leave.</p>
<p>Games I played:</p>
<ul>
<li><a href="http://en.wikipedia.org/wiki/Carcassonne_%28board_game%29">Carcassonne</a>, which I’ve played before.</li>
<li><a href="http://www.alderac.com/loveletter/">Love Letter</a>, I’ve heard of it but never played. Enjoyed it.</li>
<li><a href="http://boardgamegeek.com/boardgame/70323/king-tokyo">King of Tokyo</a> a very amusing game for 4-6 players</li>
</ul>
<h3 id="wrap-up">Wrap-up</h3>
<p>As usual, the videos from PyCon are posted online very quickly. All of these
talks can be found on the <a href="https://www.youtube.com/channel/UCgxzjK6GuOHVKR_08TT4hJQ">PyCon 2015 Channel</a>.</p>
<p>Now I’m exhausted and I need to go to bed in order to be ready for tomorrow
morning’s 5k!</p>
Goodbye, Red Wheels!2014-10-21T00:00:00+00:00http://dailytechnology.net//random/2014/10/21/goodbye-red-wheels<p>While we were visiting A’s family in Dallas last week, we purchased a 2006 Honda Odyssey. Our family is outgrowing the quad-cab Tacoma, and my old 1995 Honda Civic is simply no longer practical. It only rides four, and A can’t drive it (it’s a stick-shift). Hence, it is with much sadness that I published it on Craigslist last night.</p>
<p><a href="http://daileyleaf.com/wp-content/uploads/2014/10/Picture-040-e1413943112952.jpg"><img src="http://daileyleaf.com/wp-content/uploads/2014/10/Picture-040-e1413943112952.jpg" alt="Picture 040" width="500" height="375" class="alignnone size-full wp-image-735" /></a></p>
<p>We bought the Civic in 2004 after, within six months of getting married, I totaled my SUV. The story of it’s procurement is one we still marvel at. After weeks of trolling newspaper classifieds (Craigslist didn’t have wide adoption in rural Tennessee in 2004) I found a Civic with 125,000 miles for $3300 in Chattanooga.</p>
<p>We drove an hour to go see it. The owner was an older lady (a smoker) who had bought it new. I offered her $3000.</p>
<p>“You can have it for $2800, dear!”</p>
<p>I still don’t know why she did that, but as A is my witness, that happened. Furthermore, this did not throw red flags for me (although it did for A), and I happily paid the sum.</p>
<p>It was certainly well spent. The car rolled over 201,000 miles a few weeks ago.</p>
<p>Within two weeks of owning it, I parked it in my driveway without engaging the emergency break or putting it into gear. It rolled into a tree, denting the front bumper and putting it on the straight path to cosmetic obsolescence.</p>
<p><a href="http://daileyleaf.com/wp-content/uploads/2014/10/IMG_4947_small.jpg"><img src="http://daileyleaf.com/wp-content/uploads/2014/10/IMG_4947_small.jpg" alt="IMG_4947_small" width="500" height="316" class="alignnone size-full wp-image-737" /></a></p>
<p>I slapped my “Geek Patrol” magnetic signs on it and drove it to freelance consulting gigs all over east Tennessee.</p>
<p>We first visited NYC in 2006 with my brother and my cousin. The four of us squeezed into the Civic for the 14 hour overnight drive. Your knees were pretty much at your chin. I can still see my brother wearing headphones and headbanging to metal all the way up I-81 through Virginia.</p>
<p>When we first moved to NYC, we didn’t take any vehicle with us. That lasted less than a year. I flew back, drove the Honda to NYC, and became one of the car-driving elite. We would bundle into the car with friends, go hiking in New Jersey or skiing in Pennsylvania or Vermont, and be back by Monday.</p>
<p>On one of these trips to the Catskills, I was pulled over at a checkpoint. The nice police lady asked me where I was headed.</p>
<p>“That way.” I pointed ahead.</p>
<p>“And where are you traveling from?”</p>
<p>“That way.” I pointed behind me.</p>
<p>“Have you had anything to drink?”</p>
<p>“Yep, we just ate a huge meal after a hike.”</p>
<p>“What did you drink?”</p>
<p>“Water.”</p>
<p>“… You learned that in Tennessee didn’t you.” (I still had Tennessee plates at this point.)</p>
<p>She waved me on.</p>
<p>My brother in the back-seat nearly died laughing as soon as we pulled away. He couldn’t decide if I was stupid or just trolling her.</p>
<p>One thing about this car that was great is that it was so basic. There was no power steering, no power windows, and a stick shift. There simply wasn’t much that could go wrong with this car. Of course, that had a downside, too. If I left my lights on, it didn’t beep to remind me to turn them off. So leave them on I did. I’ve been jumped off by nearly all of my co-workers, police, state troopers, National Park foresters, and random people in parking lots all over the country.</p>
<p><a href="http://daileyleaf.com/wp-content/uploads/2014/10/Picture-059-e1413943075151.jpg"><img src="http://daileyleaf.com/wp-content/uploads/2014/10/Picture-059-e1413943075151.jpg" alt="Since Brian left the headlights on, we need assistance." width="500" height="375" class="alignnone size-full wp-image-734" /></a></p>
<p>We drove it to Portland, Maine several times to visit friends in both Portland and Boston. I don’t have many pictures of the car for these trips because the car was the means, not the destination. Still, occasionally there it is, in the background (hi C and M!).</p>
<p><a href="http://daileyleaf.com/wp-content/uploads/2014/10/Picture-022-e1413943184874.jpg"><img src="http://daileyleaf.com/wp-content/uploads/2014/10/Picture-022-e1413943184874.jpg" alt="Picture 022" width="500" height="332" class="alignnone size-full wp-image-736" /></a></p>
<p>Living in Queens, I had to move it several times a week for street sweeping. The only exception was when we got a lot of snow.</p>
<p><a href="http://daileyleaf.com/wp-content/uploads/2014/10/CSC_0117-e1413943251212.jpg"><img src="http://daileyleaf.com/wp-content/uploads/2014/10/CSC_0117-e1413943251212.jpg" alt="CSC_0117" width="499" height="332" class="alignnone size-full wp-image-738" /></a></p>
<p>NYC was hard on it. People stole my license plates, tore off the window rain guards, bumped into it parallel parking, and generally wreaked destruction on its exterior. Despite how rough it looks, though, the car never left me down. I used to be terrified it would quit on me in the shoulder-less canyons of the Brooklyn-Queens Expressway, but it never did.</p>
<p>Goodbye, red wheels! You’ve served us well.</p>
<p><strong>Update (10/29):</strong> Sold the car last night for $1000 less than I bought it for 10 years ago. It’s hard to beat $100 a year or 1.3¢ per mile!</p>
CoderFaire 20142014-09-21T00:00:00+00:00http://dailytechnology.net//speaking/2014/09/21/coderfaire-maps<p>Huge kudos to Cal & Kathy Evans as well as Jacques Woodcock for organizing the
third year of CoderFaire, a language-agnostic conference that pulls all kinds of
regional talent out of the woodwork.</p>
<p>I particularly enjoyed Jason Myers’s talk: <a href="https://github.com/jasonamyers/coderfaire-net-neutrality#sources">Umm… so Yeah… Net Neutrality: Do You Know How this Internet Thing Works?</a>.
I’m not sure what I expected when I walked in the room, but it didn’t exactly
go the direction I was expecting. Not having a background in networking, it was
a crash course in how the backbone of the internet works. I entered an ignorant
rube, and left a slightly less ignorant rube.</p>
<p>I also enjoyed the impromptu panel discussion of <a href="http://12factor.net/">12 Factor
Apps</a> between <a href="http://pmjones.com">Paul Jones</a> and
<a href="https://twitter.com/jacques_thekit">Jacques Woodcock</a>. We only got through the
first six factors before running out of time, but it was interesting hearing the
audience cover different implementations across various programming stacks.</p>
<p>Each break between talks was 20 minutes long. When looking at the calendar, that
seemed like a long time, but in reality it worked out quite well. It gave enough
time for the attendees to talk, drink some coffee, and mill for a bit before
having to find seats for the next talk. I’m not sure that was intentional or
not, but it was a nice touch.</p>
<div class="pull-right text-center" style="margin: 5px;"><img class="img-rounded" src="/img/coderfaire-2014.jpg" alt="Coderfaire 2014" />
<br />
<small class="text-muted">Credit: <a href="http://jasonmyers.com">Jason Myers</a></small>
</div>
<p>I was happy to give a short talk on mapping geospatial data for the web. I gave
a similar talk at Barcamp two years ago, and the tools available have changed
significantly since then.</p>
<p>Here are the resources from my talk on mapping geosptial data on the web.</p>
<ul>
<li><a href="http://dailytechnology.net/2014-coderfaire-maps/">Slides</a>
<ul>
<li>Press ‘s’ when opening them up to see speaker notes.</li>
</ul>
</li>
<li><a href="https://github.com/briandailey/2014-coderfaire-maps/">Source Code</a></li>
<li><a href="http://dailytechnology.net/2014-coderfaire-maps/examples/leaflet.html">Example Leaflet Sample Map</a></li>
<li><a href="http://dailytechnology.net/2014-coderfaire-maps/examples/d3.html">Example D3 Sample Map</a></li>
<li>Other Resources
<ul>
<li><a href="https://data.nashville.gov/Development-and-Housing/Building-Permits/3h5w-q8b7">Nashville Building Permits</a></li>
<li><a href="http://www.bostongis.com/printerfriendly.aspx?content_name=ogr_cheatsheet">ogr2ogr cheatsheet</a></li>
<li><a href="http://pyvideo.org/video/672/making-maps-with-python">Making Maps with Python (Trulia, 3 hour tutorial 2012 P</a></li>
<li><a href="http://bost.ocks.org/mike/map/">d3: Let’s Make a Map</a></li>
<li><a href="http://chimera.labs.oreilly.com/books/1230000000345/ch12.html">Interactive Data Visualization for the Web (Chapter 12)</a></li>
<li><a href="http://colorbrewer2.org/#">Colorbrewer</a></li>
<li><a href="https://pinboard.in/u:kungfoofool/t:mapping/">My Mapping Bookmarks (Pinboard)</a></li>
</ul>
</li>
</ul>
The Rudeness Test2014-07-16T00:00:00+00:00http://dailytechnology.net//nashville/2014/07/16/the-rudeness-test<p>At one of the recent user groups, I had an interesting conversation with Nick, a
senior developer from San Francisco who is responsible for hiring a remote team
here in Nashville.</p>
<p>Developers have a reputation for being difficult to work with, and this is
especially true in coastal cities. As part of his interview process, Nick
included a filter test for rudeness.</p>
<p>One of the measures for testing this is that Nick will interrupt an interviewee
in the middle of a discussion. If the candidate continues talking over Nick,
that’s generally a bad thing. If the candidate stops and politely waits for Nick
to have his say, that indicates someone who is willing to give way when
necessary.</p>
<p>According to Nick, no one in Nashville failed the rudeness test. In fact, Nick
is now concerned that he needs a different approach for folks in Nashville. How
can he be sure that they are aggressive enough that they will interrupt someone
if they drone on and on?</p>
<p>As an aside: when I lived in New York for a short while, this cultural
difference was a tough one for me (a native Southerner) to adjust to. There were
many situations in a group where one person would start talking over another.
Meanwhile, the original speaker, unwilling to give up the floor, would continue
talking. I generally try to make eye contact with whoever is speaking to me, so
if two people are talking at once I’m trying to figure out who I should be
looking at and paying attention to.</p>
<p>Of course, this is a very simplified view of cultural and regional norms (I
certainly learned to appreciate the bluntness of friends from NYC), but it’s
still an interesting illustration of how different cultures can have different
issues when hiring for a cultural fit.</p>
Nashville Open Data Initiative2014-05-14T00:00:00+00:00http://dailytechnology.net//nashville/2014/05/14/nashville-open-data<p>On May 12, 2014 Nashville’s Mayor Karl Dean signed <a href="http://www.nashville.gov/Metro-Clerk/Legal-Resources/Executive-Orders/Mayor-Karl-Dean/kd043.aspx">Executive Order #43</a>.
Thanks to the sartorially gifted <a href="https://twitter.com/jacques_thekit">Jacques
Woodcock</a> I was able to attend the event
in order to help answer questions from the media about how this is a Good
Thing.</p>
<p>In general, any level of transparency from government is a Good Thing. However,
some common issues that arise with “open data” is as follows:</p>
<ul>
<li>The data is often released in unfriendly formats (e.g., PDF, Word Documents)
<ul>
<li>This means the data is often summarized, so you get no details about the
underlying data.</li>
<li>It’s also more difficult to parse the data in any automated way</li>
</ul>
</li>
<li>The data is not timely enough to be useful.
<ul>
<li>Obviously, historical data can be useful in its own right, but as a
general rule, the more timely the data, the more valuable it is.</li>
</ul>
</li>
</ul>
<p>In the past, this has certainly been an issue with data released from Metro
Nashville. However, the Executive Order is intended to address some of these
issues.</p>
<h3 id="the-good">The Good</h3>
<h4 id="the-objectives">The Objectives</h4>
<p>First, the order has the following admirable objectives:</p>
<ul>
<li>Publish Open Data in a timely and consistent manner.</li>
<li>Publish Open Data in machine-readable formats that engage the community by
soliciting their ideas, input, and creative energy to support the development
of applications and tools that enable the public to access, visualize, and
analyze Open Data in new and innovative ways.</li>
<li>Adopt standards that improve transparency, access to Open Data, and improved
coordination and efficiencies across Metropolitan Government departments.</li>
</ul>
<p>This addresses the two issues mentioned above. It’s difficult to be precise
about what the adopted “standards” will be, but I assume it means that each
data set should have a clearly defined process for pushing it to Socrata.</p>
<h4 id="the-platform">The Platform</h4>
<p>That leads me to the next good thing: Socrata. Socrata is the platform that
Metro has chosen to host their data. It’s a common choice for municipalities.
New York, Chicago, and others have already been publishing their data sets on
the platform. It is, alas, a for-profit entity so there is some risk associated
with choosing it as a platform. From what I gathered in conversations with
Metro’s Chief Innovation Officer <a href="http://www.nashville.gov/Government/Staff-Directory/Staff-Member-Details/ID/377/Yeh-Yiaway">Yiaway
Yeh</a>
the platform is paid for with federal grants. Hopefully the continuity of the
underlying grants is not in question (if you have more information on this,
please leave a comment at the bottom!).</p>
<p>On the other hand, open source projects have a huge issue with long-term
support. Projects that cannot be maintained internally suffer from abandonment.
Any project that expects long-term viability must be absolutely dead-simple to
host and maintain (a great example of this is the <a href="http://www.eslmap.com/">ESL
Map</a> written at HackNashville and the work done it by
my colleague <a href="http://twitter.com/jasonamyers">Jason Myers</a>). Perhaps hosting it
with a for-profit company has it’s upside in long-term viability.</p>
<p>Socrata platform’s wide adoption makes it an overall good choice, in my opinion.
It’s <a href="http://dev.socrata.com/">well-documented</a>, for starters.</p>
<h3 id="the-not-so-great">The Not-So-Great</h3>
<p>I’m not terribly pleased with the list of exempted departments. It’s a rather
long list with some fairly large departments:</p>
<blockquote>
<p>Board of Education/Metropolitan Nashville Public Schools, Nashville Electric
Service, Metropolitan Nashville Airport Authority, Metropolitan Development
and Housing Agency, Metropolitan Transit Authority, Metropolitan Sports
Authority, Convention Center Authority, Health and Educational Facilities
Board, Industrial Development Board, non-professional employees of the Board
of Health, and the offices of elected officials. The Mayor requests,
however, that the excluded entities voluntarily undertake to develop and
adopt similar policies.</p>
</blockquote>
<p>That last touch is nice, but I wonder if it will carry much weight.</p>
<h3 id="lets-build-something">Let’s Build Something!</h3>
<p>The timing of the order coincides with the <a href="http://hackforchange.org/">National Day of Civic
Hacking</a> and it’s local branch, <a href="http://hack4changenash.org/">Hack For Change
Nashville</a>. I definitely look forward to seeing
what local developers do with this data!</p>
<p>What data sets do you find most interesting? What questions do you think
this data might answer? Leave a comment or get in touch with me on Twitter
at <a href="http://twitter.com/byeliad">@byeliad</a>.</p>
<h3 id="resources">Resources</h3>
<ul>
<li><a href="http://www.nashville.gov/Government/Open-Data.aspx">List of Open Data with Metadata</a></li>
<li><a href="http://www.nashville.gov/Metro-Clerk/Legal-Resources/Executive-Orders/Mayor-Karl-Dean/kd043.aspx">Executive Order #43</a></li>
<li><a href="https://data.nashville.gov/">Data Hosted on Socrata</a></li>
<li><a href="http://dev.socrata.com/">Socrata API Documentations</a></li>
<li><a href="http://www.nashville.gov/News-Media/News-Article/ID/2837/Mayor-Signs-Executive-Order-to-Publish-Open-Data.aspx">Press Release: Mayor Signs Executive Order to Publish Open Data</a></li>
<li><a href="http://hack4changenash.org/">Hack for Change Nashville</a></li>
</ul>
Interactive Developers Nashville: Command Line Kung Fu2014-03-19T00:00:00+00:00http://dailytechnology.net//speaking/2014/03/19/interactive-developers-nashville-command-line-kung-fu<p>Last night I presented <a href="http://www.meetup.com/interactivedev/events/135589052/">“Command Line Kung Fu” to the Interactive Developers of
Nashville</a> group in Brentwood. I enjoyed the discussion around the Unix
philosophy of modularity and how the various tools complement one another to do
powerful things.</p>
<div class="pull-right span4">
<img src="/assets/themes/dailytechnology/img/meetup_cli_talk.jpg" alt="Photo by Matthew Wallace" />
<br />
<small>Photo by Matthew Wallace</small>
</div>
<p>This is a talk I’ve actually given a <a href="/talks/2009/04/25/nyphp-presentation-become-a-bash-ninja">few</a>
<a href="http://www.slideshare.net/brian_dailey/devchatt-2010-nix-cmd-line-kung-foo">times</a>
<a href="/talks/2009/10/23/command-line-kung-fu-at-ruby-on-rails-october-meeting">before</a> with
some variations.</p>
<p>I appreciate ID hosting me for the evening, and I hope it was useful to the
attendees! Thanks to all of those that were able to make it, and thanks to my
colleague <a href="http://www.jasonamyers.com/">Jason Myers</a> for lending moral support.</p>
<p>Slides from the talk are available
<a href="http://dailytechnology.net/talk-cli-intro/?theme=sky">here</a>. If you want to
see my notes for the talk, you can press the ‘s’ key to open the speakers
notes. As usual, I used <a href="https://github.com/hakimel/reveal.js">reveal.js</a> for
building the presentation itself.</p>
On Award Ceremonies2014-02-04T00:00:00+00:00http://dailytechnology.net//2014/02/04/why-i-dont-follow-awards<p>In business, there are a lot of awards. There are a lot of organizations that
spend time “recognizing” business leaders for their various accomplishments.
One assumes are great for company publicity, I suppose, but how effective are
they?</p>
<p>I’ve been preparing my talk on <a href="http://pytennessee.tumblr.com/post/75150059974/pytn-profiles-brian-dailey-and-adafruit">Systematic Biases and the Software Developer</a> for
<a href="http://pytennessee.org">PyTennnessee</a> and I ran into research that would appear to be relevent.</p>
<p>Ulrike Malmendier and Geoffrey Tate wrote a paper titled <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=972725">
Superstar CEOs</a>. The following quote
was included in <a href="http://www.amazon.com/gp/product/0374533555/ref=as_li_ss_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=0374533555&linkCode=as2&tag=rea036-20">Thinking, Fast and Slow</a>.</p>
<blockquote>
<p>“We find that firms with award-winning CEOs subsequently underperform, in
terms both of stock and operating performance. At the same time, CEO
compensation increases, CEOs spend more time on activities outside the company
such as writing books and sitting on outside boards, and they are more likely
to engage in earnings management.”</p>
</blockquote>
<p>I was amused to find many of the award nomination forms I’ve seen include
several questions about which boards CEOs sit on.</p>
Mac OS X Mavericks and Android File Transfer2013-11-07T00:00:00+00:00http://dailytechnology.net//random/2013/11/07/mac-os-x-maverick-and-android-file-transfer<p>If you have recently upgraded to Mac OS X Mavericks and you discover that you can no longer detect your Android phone with Android File Transfer, this should fix it for you.</p>
<ul>
<li>Open up Settings.</li>
<li>Choose Storage.</li>
<li>On the top right is an option dropdown, hit that and hit “USB computer connection.”</li>
<li>If MTP is unchecked (as mine was), AFT won’t detect your device. Enable it.</li>
<li>Reconnect and everything should work properly.</li>
</ul>
<p>I haven’t messed with this setting in the past so I don’t know why it was unchecked, but apparently something in the Mavericks upgrade affected the ability of AFT to detect the phone without MTP enabled.</p>
<p>Hopefully this will help out other frustrated hybrid Mac OS X and Android users.</p>
Amazon Redshift - What You Need To Know2013-08-03T00:00:00+00:00http://dailytechnology.net//2013/08/03/redshift-what-you-need-to-know<p>Stratasan, a company I co-founded in 2010, moved from PostgreSQL to Amazon
Redshift in 2013. Now that we’ve migrated most of our data to the platform, I
thought it would be useful to others out there if I summarized some of our
experiences during the transition (much like <a href="http://nerds.airbnb.com/redshift-performance-cost">others have
done</a> before us).</p>
<h3 id="what-is-amazon-redshift">What Is Amazon Redshift?</h3>
<p>Amazon Redshift is a managed database platform that looks, smells, and tastes
an awful lot like PostgreSQL 8. That said, it isn’t really PostgreSQL. Before
Amazon released Redshift, a company called
<a href="http://www.zdnet.com/amazon-redshift-paraccel-in-costly-appliances-out-7000008111/">ParAccel</a>
had built a managed database platform that was designed for analytics. What set
it appart is that it was originally touted to run well on commodity hardware.
(Source: <a href="http://en.wikipedia.org/wiki/ParAccel">Wikipedia</a>) Amazon made a
significant investment in the company, and used their platform to create
Redshift.</p>
<p>Redshift is built for analytics. It’s a columnar data store, so it’s intended
to return fast results for aggregations (SUM, AVG, COUNT, etc). Single-store
lookups are, of course, much slower.</p>
<h3 id="why-redshift">Why Redshift</h3>
<p>We checked out Hadoop and Elasticsearch prior to using Redshift. For various
reasons, neither fit our need very well. We needed a way to handle data in
various shapes and sizes without spending a ton of time on pre-processing. As a
Python shop, we didn’t want to spend an inordinate amount of time learning Java
or Clojure.</p>
<p>Redshift smelled enough like PostgreSQL that we felt like we could move our
existing PostgreSQL-stored data to it and query it using the same ad-hoc tools
we had already built. Most of our data usage comes from aggregate queries, so
it seemed to perfectly fit our need. The ability to reuse both our existing code
and knowledge with PostgreSQL seemed like a no-brainer.</p>
<h4 id="cost">Cost</h4>
<p>Part of the reason I like Redshift is the cost. As the co-founder of a small
startup, I have to be able to communicate effective use of our funds to
interested parties. As someone at the <a href="http://blog.aggregateknowledge.com/2013/05/16/aws-redshift-how-amazon-changed-the-game/">Aggregate Knowledge
blog</a>
pointed out:</p>
<blockquote>
<p>Oddly enough, Redshift isn’t going to sell because devs think it’s
super-duper-whizz-bang. It’s going to sell because it took a problem
and an industry famous for it’s opaque pricing, high TCO, and unreliable
results and completely turned it on its head.</p>
</blockquote>
<p>Amazon pricing model is quite clear:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>XL = 0.85/hr
8XL = 8 x 0.85/hr = $6.80/hr
</code></pre></div></div>
<p>If you buy a reserved instance, the price goes down:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Unreserved: $3,723 per TB per Year
Reserved 1 year: $2,190 per TB per Year
Reserved 3 years: $999 per TB per year
</code></pre></div></div>
<h4 id="speed">Speed</h4>
<p>One of the issues we ran into is that we could not always anticipate what data
would look like prior to receiving it. We didn’t want to spend an inordinate
amount of resources throwing indexes on columns that may or may not be used.
Therefore, most of our ad hoc queries were against completely unindexed data.</p>
<p>Since Redshift doesn’t use indices, this wasn’t an issue.</p>
<p>As a preface, any benchmark I show you is going to be somewhat arbitrary. To
properly measure this, you’d have to use an EC2 instance with properties
similar to the Redshift architecture. I didn’t have time to do that, so what
I’m actually going to show you is what our “real world” speed increase looked
like.</p>
<p>In this example, I’m using a table that we loaded up onto a dedicated Dell
server running PostgreSQL 9.1.9.</p>
<p>Here’s a SELECT COUNT on the table in PostgreSQL:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># select count(*) from dummy_table;
+----------+
| count |
+----------+
| 21454134 |
+----------+
(1 row)
Time: 586931.216 ms
</code></pre></div></div>
<p>That took approximately 10 minutes. And on Redshift:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># select count(*) from dummy_table;
count
----------
21454134
(1 row)
Time: 1561.359 ms
</code></pre></div></div>
<p>One and a half seconds! That’s an improvement. Note that I didn’t make any adjustements to
the data: no indexes, no differences in table structure.</p>
<p>How about some aggregates?</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># select count(*), year, avg(total_charges) \
from dummy_table group by year order by year desc;
+---------+------+---------+
| count | year | avg |
+---------+------+---------+
| 4926225 | 2012 | 6144.37 |
| 8280484 | 2011 | 6149.85 |
| 8247425 | 2010 | 5673.83 |
+---------+------+---------+
(3 rows)
Time: 850491.316 ms
</code></pre></div></div>
<p>Approximately 15 minutes. Ouch. And Redshift:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># select count(*), year, avg(total_charges) \
from dummy_table group by year order by year desc;
count | year | avg
---------+------+---------
4926225 | 2012 | 6144.37
8280484 | 2011 | 6149.85
8247425 | 2010 | 5673.83
(3 rows)
Time: 5887.693 ms
</code></pre></div></div>
<p>Not bad! Definitely an improvement. There are some things we could do to make
this even faster (in both cases), but that’s not really the focus here. How
much work did I have to do to prepare this data for a relatively fast aggregate
query? None at all.</p>
<h3 id="redshift-architecture">Redshift Architecture</h3>
<p>If you log into the AWS console, the first thing you’ll do is create a
<em>cluster.</em> Each cluster can have one or more Redshift databases.</p>
<p>A Redshift cluster consists of <strong>leader</strong> nodes and <strong>compute</strong> nodes.</p>
<p>A <strong>leader</strong> node is only created if there are two or more nodes. It is the
node that you interact with. It takes care of the planning, compiles queries
into code, and distributes it to the compute nodes. It’s also where you’ll find
all of the pg_ catalog tables (e.g., pg_views, pg_table_def).</p>
<p>If for some reason you run a SELECT query that doesn’t actually touch data, it
will run on the leader node. One example of this is:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>select now()
</code></pre></div></div>
<p>As of August 2013, the <strong>compute</strong> nodes come in two flavors:</p>
<ul>
<li>XL: 2 Cores, 15GiB memory, 3 disk drives with 2TB of lcoal attached storage</li>
<li>8XL: 16 Cores, 120GiB memory, 24 disk drives with 16TB of lcoal attached storage
You can scale up your database by adding or removing compute nodes.</li>
</ul>
<p>Each compute node consists of multiple slices. When you send a query to the
leader node, it compiles the query and sends it to each node. Each node
executes the query in parallel on each slice (there is one slice for each core
of the multi-core processor).</p>
<p>In between the leader node and all of the compute nodes is a 10GigE HPC
network. That keeps the network traffic between all of the machines snappy.</p>
<h3 id="building-tables">Building Tables</h3>
<p>Before you can load anything into Amazon Redshift, you’ll have to build some
tables to store it in. This it the first thing you’ll do that causes you to
notice some of the differences between Redshift and PostgreSQL.</p>
<p>First, Redshift “does not enforce unique, primary-key, and foreign-key
constraints.” Your application code is responsible for guaranteeing uniqueness.
That being said, you will still want to add CONSTRAINT indicators to your
table. Redshift uses it to optimize its queries
(<a href="http://docs.aws.amazon.com/redshift/latest/dg/c_best-practices-defining-constraints.html">Source</a>).</p>
<p>Secondly, if you’re migrating from an existing daatabase, you’ll notice that
there are a lot of unsupported PostgreSQL data types. The ones that hurt the
most for us was SERIAL, since we used that for primary key columns.
Arrays, JSON, XML - all unsupported.</p>
<p><strong>Update</strong> As of September 2013 JSON is now supported. Thanks <a href="https://twitter.com/rahulpathak/status/384425227934892033">@rahulpathak</a>.</p>
<p>It’s actually easier to list what <em>is</em> supported.</p>
<ul>
<li>SMALLINT</li>
<li>INTEGER</li>
<li>BIGINT</li>
<li>DECIMAL</li>
<li>REAL</li>
<li>DOUBLE PRECISION</li>
<li>BOOLEAN</li>
<li>CHAR</li>
<li>VARCHAR</li>
<li>DATE</li>
<li>TIMESTAMP</li>
</ul>
<p>One huge gotcha in this list - if in your CREATE statement you provide a length
to VARCHAR or CHAR, note that it is byte length, not character length. This
only bites you if you store multi-byte data, but it’s an important thing to
aware of when defining your data storage. Furthermore, Redshift only
accommodates UTF8 characters up to 3 bytes long. If you’re doing a lot of
internationalization, this is something you very much want to pay attention to.</p>
<p><strong>Update</strong> As <a href="https://twitter.com/rahulpathak/status/384425227934892033">@rahulpathak</a> pointed
out, UTF8 up to four bytes is now supported.</p>
<h4 id="sort-and-distribution-keys">Sort and Distribution Keys</h4>
<p>There are two other things you want to familiarize yourself with when creating
tables on Redshift.</p>
<p>By default, the contents of your table are distributed across your nodes and
slices in a round-robin fashion. Evenly distributing the data means that the
slices can all share equal amounts of work. If all of the data was stored in
one slice, you couldn’t take advantage of Redshift’s use of parallel queries.</p>
<p>However, you can indicate your own <em>distribution key</em>. An ideal candidate is any
column that you commonly group by (for aggregates) or perform table joins on.
You still want to keep the data as evenly distributed as possible, however.</p>
<p>An example of this would be a table with an “favorite_color” column. If the
values are pretty evently distributed, then grouping them together means
Redshift has to do less work in copying data around inside the cluster to
perform the aggregate. On the other hand, if 90% of your table data has a
favorite_color of “red”, then the node or slice with the “red” data is going to
have to do a disproportionate amount of work.</p>
<p>The <em>sort key</em> indicates how your data should be sorted on disk. You can
indicate multiple sort keys for a given table.</p>
<p>Here’s the important thing to know:</p>
<blockquote>
<p>“Amazon Redshift stores columnar data in 1 MB disk blocks. The min and max
values for each block are stored as part of the metadata.”</p>
</blockquote>
<p>This means that if you have a sort key indicated, Redshift can skip over swaths
of your data to pinpoint exactly where the value you’re looking for can be
found.</p>
<p>Amazon’s webinars often use the example of a “last updated timestamp” column to
demonstrate how this works. If you are most commonly accessing data based on
that timestamp you can use it as a sort key. When searching for values, it will
only look at nodes, slices, and blocks that are relevant.</p>
<h3 id="alter">ALTER</h3>
<p>One last thing to know about building tables. If you want to ALTER a column
after the table has been created, you’re <a href="http://docs.aws.amazon.com/redshift/latest/dg/c_redshift-sql-implementated-differently.html">out of
luck</a>.
If you want to, for example, change type or length, you’ll need to ADD another
column, copy the data over, and remove the old column.</p>
<p>Additionally, you can only ALTER ADD COLUMN one at a time. So if you need to do
this multiple times, you’re much better off creating a new table and copying
the data into it. For example:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>COPY (SELECT * FROM old_table) INTO new_table;
</code></pre></div></div>
<h3 id="loading-data">Loading Data</h3>
<p>So now that we’ve created out data structure, how do we get data into it? You
have two choices:</p>
<ul>
<li>Amazon S3</li>
<li>Amazon DynamoDB</li>
</ul>
<p>Yes, you <em>could</em> simply run a series of INSERT statements, but that is going to
be painfully slow.</p>
<p>Amazon recommends using the S3 method, which I will describe briefly. I don’t
see the DynamoDB as particularly useful unless you’re already using that and
want to migrate some of your data to Redshift.</p>
<p>To get the data from your local network to S3, I would use
<a href="http://s3tools.org/s3cmd">s3cmd</a>. It allows you to break the file into
multiple pieces for upload. Another option is to use a package like
<a href="https://github.com/boto/boto">boto</a>. Although it requires more work, you can
stream files concurrently rather than one at at time. As of version 1.1, s3cmd
does not allow you to load files in paralell.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>s3cmd put /path/to/my/file.sql.gz* s3://my-bucket/
</code></pre></div></div>
<p>There are a few things to note here. First, if you can zip your files before
uploading them, that will save you some time and bandwidth. Second, if you
break them up into multiple chunks, Redshift can load them in parallel from S3.
Note the asterisk after the file name: this assumes the file is chunked. For
example:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>file.sql.gz.0001
file.sql.gz.0002
</code></pre></div></div>
<p>To get the files from S3 to Redshift, you will use the
<a href="http://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html">COPY command</a>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>COPY my_table (column1, column2) FROM 's3://my-bucket/file.sql.gz' WITH
CREDENTIALS '...' GZIP
</code></pre></div></div>
<p>Note that the s3 path is a prefix, so it will grab all files that begin with
‘file.sql.gz’ and load them in parallel.</p>
<p>Of course, you may run into some problems at this point if your data doesn’t
match the structure you’re loading it into. In that case, the following tables
can be checked for details:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>STL_LOAD_ERRORS
STL_LOADERROR_DETAIL
</code></pre></div></div>
<p>It is generally a good idea to run the COPY command with the NOLOAD option
first, so you can scan the entire file for any issues. It has the advantage of
running more quickly than an actual load, and you can also bump up the MAXERROR
to continue finding more problems as you scan the file. Just remember to reset
MAXERROR when you’re done!</p>
<h3 id="querying-data">Querying Data</h3>
<p>The beauty of Redshift is that if you’ve ever used PostgreSQL, you already know
how to query it. The syntax is nearly identical. Remember to use Postgres 8.0.2
documentation, however, and always keep the <a href="http://docs.aws.amazon.com/redshift/latest/dg/c_redshift-sql-implementated-differently.html">implementation
differences</a>
handy.</p>
<h3 id="extracting-data">Extracting Data</h3>
<p>If you want to dump data from Redshift, you will use the
<a href="http://docs.aws.amazon.com/redshift/latest/dg/r_UNLOAD.html">UNLOAD</a> command.
Note that UNLOAD will dump your query into multiple files with the prefix you
provide. This will be based on the number of slices in your cluster, so even
small queries will create multiple files. There is, however, a <a href="https://forums.aws.amazon.com/message.jspa?messageID=432538">way to
trick</a>
UNLOAD into dumping everything into one file by adding an outer LIMIT.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>UNLOAD 'SELECT * FROM (
SELECT ...
FROM ...
WHERE ...
) LIMIT 2147483647'
...
</code></pre></div></div>
<p>It does, however, affect performance.</p>
<h3 id="optimizations">Optimizations</h3>
<p>You’ll want to read up on the <a href="http://docs.aws.amazon.com/redshift/latest/dg/c_workload_mngmt_classification.html">workload manager</a> (WLM).
It allows you to create lanes for parallel queries, which means you can prevent long-running
queries from hogging resources.</p>
<blockquote>
<p>“By default, a cluster is configured with one queue that can run five
queries concurrently. In addition, Amazon Redshift reserves one dedicated
Superuser queue on the cluster, which has a concurrency level of one.
The Superuser queue is not configurable.”</p>
</blockquote>
<p>You can create up to 8 queues, and you can allow up to 15 concurrent queries.
One strategy is to set the concurrent queries to 1 when you are running a COPY
statement. That speeds up the load at the cost of making other queries wait.
You can also assign queries at runtime, or set them up by user.</p>
<p>When you load data, VACUUM is automatically run. However, if you make any non-trivial
changes to your data (UPDATE, DELETE, or INSERT) you will want to run it again
with ANALYZE to resort the data.</p>
<p>Other optimizations are highly dependent on the data you’re loading. One
huge advantage of Redshift is that you can try multiple configurations (e.g.,
few large nodes vs many small notes, 16xXL versus 2x8XL) at a low cost.</p>
<h3 id="wrap-up">Wrap-up</h3>
<p>The advantages of Redshift to any shop wishing to run adhoc queries against large
sets of data is abundantly clear.</p>
<ul>
<li>It looks and smells like PostgreSQL 8.</li>
<li>It’s much easier to use existing talent than learning to use a new tool like Hadoop.
<ul>
<li>As others have pointed out, Hadoop is generally overutilized anyway.</li>
<li>Even AirBNB analysts liked it so much they didn’t want to go back to PIG and HIVE.</li>
</ul>
</li>
<li>It’s less expensive than appliances like Vertica.</li>
<li>It’s pricing scheme is much more clear than competing appliances.</li>
</ul>
<p>Making the switch from PostgreSQL to Redshift has not only made our adhoc queries faster,
it’s also saved us hosting costs.</p>
<p>If you have any questions about our implementation, I would be happy to talk about it
further. If you’re in the Nashville area, I’m always up for grabbing coffee or lunch. Feel
free to email me (brian at this domain) or ping me on Twitter at @byeliad.</p>
<p>This blog post is based on my notes from a talk I gave at Coderfaire 2013. Slides are
<a href="http://dailytechnology.net/talk-coderfaire-redshift-2013/">available here</a>.</p>
<h3 id="additional-resources">Additional Resources</h3>
<ul>
<li><a href="http://www.slideshare.net/AmazonWebServices/redshift-best-practices-part-1dp2">AWS Webcast - Amazon Redshift Best Practices for Data Loading and Query Performance</a></li>
<li><a href="https://amplab.cs.berkeley.edu/benchmark/">Big Data Benchmark</a> from Berkeley, comparing Redshift, Hive, and others.</li>
</ul>
My LinkedIn Rule2013-08-03T00:00:00+00:00http://dailytechnology.net//random/2013/08/03/my_linkedin_rule<p>I have a silly rule that I apply to my LinkedIn network: If I haven’t met you in person at least once, I will request to meet you in person at least once before adding you to my network.</p>
<p>To me, this makes the difference in using LinkedIn as a real connection tool, versus using it as a simple rolodex of people I may or may not know.</p>
<p>I find it somewhat disappointing that roughly half of these requests go completely dark when I request coffee or lunch. If you’ve received a link to this post after adding me on LinkedIn, I encourage you to take me up on my offer and schedule a time for us to grab coffee.</p>
Bringing Talent To Nashville: Swag Isn't The Answer2011-06-08T00:00:00+00:00http://dailytechnology.net//random/2011/06/08/bringing-talent-to-nashville-swag-isnt-the-answer<p>This post is in response to a recent post by Nashville’s venerable Dave Delaney (founder of Geek Breakfast, et al). In his post, Dave proposes that Nashville creates a position of “Technology Ambassador” to promote the community we have here in Nashville to a wider audience.</p>
<p>I can only speak from my experience, but when my family relocated from NYC to Nashville in 2009, it wasn’t swag-bearing social media experts that convinced me to to do so. In fact, I’m pretty sure if I had been assaulted by an over-eager ambassador at a conference with branded tote-bags, lapel buttons, and t-shirts, I would have written it off as a middle America city that was simply trying much too hard.</p>
<p>When I was deciding where to relocate the criteria came down to three basic questions:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>What is the quality of life like in the community? (Is traffic horrific? Do people bike to work?)
What is the cost of living and the average salary for someone with my level of experience?
What does the tech community look like (language user groups, book discussions)?
</code></pre></div></div>
<p>The first two are obviously much more important. I didn’t want to live in Atlanta and deal with it’s horrific traffic. I didn’t move to North Carolina because I wanted to live in a state with a low cost of living. Both of those cities pay more, on average, for developers, but the deciding factor for me was quality of life.</p>
<p>As for the third criteria, Nashville has a handful of high-level “technologist” functions (NTC, Digital Nashville, Geek Breakfast), but more importantly, it also has a contingent of more in-depth user groups (Nashville PHP, Nashville Dynamic Languages, .NET Nashville, Arduino Nashville, and many, many others). Those are important, and any city worth its digital salt needs those groups to grow.</p>
<p>Geek Breakfast, in particular, was instrumental in getting me connected to the larger tech community. Through it I found out about the Centresource Mixer. There I met Nicholas Holland, from whom I rented office space to continue plying my trade. While working there, I met Jason Moore (a fellow tenant), who talked me into joining him in a new venture in the healthcare sector. That company recently “graduated” from the Entrepreneur Center downtown, received funding, and is now chugging along full steam.</p>
<p>The fact that in two years I could go from a Nashville newbie to company co-founder is a testament to the vibrant community here.</p>
<p>It wasn’t marketers bearing swag.</p>
<p>If we really want to make Nashville a more attractive city to smart, talented, and creative people, we need to focus our efforts on increasing the quality of life. We need to elect representatives who are more focused on improving our city’s infrastructure, not criminalizing the sharing of Netflix passwords. We need to elect leaders who care about reducing pedestrian deaths, or increasing awareness of public transit, not leaders who are preoccupied with publication of images that might offend someone. Our fair city has less green space than the average city of comparable size. It’s mass transit is unreliable. It tops the list of cities in both pedestrian deaths and forcible rapes. [Source]</p>
<p>It would take a mountain of swag to overcome the provincial, xenophobic sort of legislation for which our state makes national news.</p>
<p>Instead of passing our city off as something it isn’t (it is not Silicon Alley and it never will be Silicon Alley - for some people this is a good thing), we should acknowledge its warts, appreciate its advantages and sense of community, and work on changing the parochial politics and increasing the city’s quality of life.</p>
Best Buy Secret Santa Featured In Ad Age2009-12-10T00:00:00+00:00http://dailytechnology.net//press/2009/12/10/best-buy-secret-santa-featured-in-ad-age<p>My consulting firm, Realm3 LLC, served as a Development Partner with Crispin, Porter + Bogusky to develop a Facebook integrated version of the popular Secret Santa game for Best Buy.</p>
<p>Ad Age wrote about retailers embracing social media to reach their target audiences, and included the application as a demonstration of a retailer “getting it right.”</p>
<div class="pull-left span5"><img align="left" src="/img/bestbuy-facebook.jpg" alt="Screenshot of Best Buy application." /></div>
<blockquote>
<p>The retailer also has a robust Facebook presence, featuring several exclusive applications such as Hint Helper, Secret Santa, Idea Giftr and, coming soon, Christmas Morning Simulator. The Hint Helper will drop personalized hints to friends and family via a cookie placed on their computers. While Best Buy’s application section is robust, it could do a better job of leading the conversation on Facebook, as opposed to simply responding to customer complaints. Still, when it comes to customer interaction, Best Buy has it down on Twitter. Its handles @bestbuy and @twelpforce, not to mention CMO Barry Judge’s own handle, @bestbuycmo, all do a stellar job.</p>
</blockquote>
<p>CPB did a great job designing this application, and I’m glad I could be a part of it’s development.</p>
Command Line Kung Fu at Ruby on Rails October Meeting2009-10-23T00:00:00+00:00http://dailytechnology.net//talks/2009/10/23/command-line-kung-fu-at-ruby-on-rails-october-meeting<p>A lot of the material I went over was from the NYPHP presentation that I gave earlier this year, but I did spend a little more time on functionality specific to the bash prompt.</p>
<p>A couple of the things that seemed to generate the most enthusiasm:</p>
<ul>
<li>Using "cd -" to toggle between the most recently viewed directories.</li>
<li>The use of GNU screen.</li>
<li>Using aliases, especially for aliasing mysql --safe-updates</li>
</ul>
<div style="margin: 0 10px 10px 0; float: left;">
<a href="http://www.flickr.com/photos/kungfoofool/4037364700/"><img src="http://farm3.static.flickr.com/2685/4037364700_bb3ce72afd_m.jpg" border="0" /></a>
</div>
<p>Some of the attendees had their own useful suggestions. Daniel Nelson pointed out that Ctrl+N and Ctrl+P is faster than using arrow keys to navigate through your command history. He also pointed out that you could pass command output into arguments using parenthesis. For example:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">diff <<span class="o">(</span><span class="nb">ls</span> <span class="nt">-l</span><span class="o">)</span> <<span class="o">(</span><span class="nb">ls</span> ../ <span class="nt">-l</span><span class="o">)</span></code></pre></figure>
<p>That's pretty slick if you ask me!</p>
<p>A lot of PHP guys showed up, so there was some interest in a small function I wrote to parse out information from the php.net information pages. Here it is in it's full glory:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">phpargs<span class="o">()</span> <span class="o">{</span>
curl <span class="nt">-s</span> http://us3.php.net/<span class="nv">$1</span> | <span class="se">\</span>
<span class="nb">sed</span> <span class="nt">-n</span> <span class="s1">'/<div class="methodsynopsis dc-description">/,/<\/div>/p'</span> | <span class="se">\</span>
<span class="nb">sed</span> <span class="s1">'s/<[^>]*>//g'</span> | <span class="nb">tr</span> <span class="nt">-d</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span>
<span class="nb">echo</span>
<span class="o">}</span></code></pre></figure>
<p>It's pretty basic, but if you add this to your bash profile you can run it by passing a function name.</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">phpargs strpos
int strpos <span class="o">(</span> string <span class="nv">$haystack</span> , mixed <span class="nv">$needle</span> <span class="o">[</span>, int <span class="nv">$offset</span> <span class="o">=</span> 0 <span class="o">]</span> <span class="o">)</span></code></pre></figure>
</pre>
<p>Again, if you're reading this and you're not checking out your local user groups, then I encourage you to do so! Even smaller cities often have user groups and they are a valuable place to both learn things and share what you know.</p>
<h3>Related Links</h3>
<ul>
<li><a href="http://www.meetup.com/nashvilleruby/">Nashville Ruby on Rails Meetup</a></li>
</ul>
Web Security 101 at BarCamp Nashville 20092009-10-02T00:00:00+00:00http://dailytechnology.net//talks/2009/10/02/web-security-101-at-barcamp-nashville-2009<strong>Edit: 10/18/2009</strong> - Here's the video and slides. The audio on the video doesn't kick in until about 9 minutes in (as I start going over framework security issues).</p>
<center>
<object classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000" width="480" height="386" id="utv648594" name="utv_n_610676"><param name="flashvars" value="loc=%2F&autoplay=false&vid=2369801" /><param name="allowfullscreen" value="true" /><param name="allowscriptaccess" value="always" /><param name="src" value="http://www.ustream.tv/flash/video/2369801" /><embed flashvars="loc=%2F&autoplay=false&vid=2369801" width="480" height="386" allowfullscreen="true" allowscriptaccess="always" id="utv648594" name="utv_n_610676" src="http://www.ustream.tv/flash/video/2369801" type="application/x-shockwave-flash" /></object>
<br/><br/>
<iframe src="http://www.slideshare.net/slideshow/embed_code/2269233" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px 1px 0; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/brian_dailey/2009-barcamp-nashville-web-security-101" title="2009 Barcamp Nashville Web Security 101" target="_blank">2009 Barcamp Nashville Web Security 101</a> </strong> from <strong><a href="http://www.slideshare.net/brian_dailey" target="_blank">brian_dailey</a></strong> </div>
<p>I'll be speaking at the Nashville BarCamp 2009, giving a brief talk on application-layer security. You can read more details on the <a href="http://barcampnashville.com/session/web-security-101">BarCamp information page</a>.</div>
NYPHP Presentation - Become A Bash Ninja2009-04-25T00:00:00+00:00http://dailytechnology.net//talks/2009/04/25/nyphp-presentation-become-a-bash-ninja<p>The objective of this presentation was to gently introduce GUI users to shell usage, and hopefully to teach some new tricks to seasoned developers.</p>
<p>I mentioned <a href="http://dotfiles.org">DotFiles.org</a> as a place to get familiar with more advanced configuration. Also check out <a href="http://commandlinefu.com/">Command Line Fu</a>.<p>
<p>I highly recommend participating in your local (insert programming preference here) user group. It's a great opportunity to learn new things or perhaps share things you've learned with others.</p>
<p>The presentation was filmed and then edited by my good friend (and excellent photographer) <a href="http://www.jamesallenwalker.com">James Allen Walker</a>. James is also working on a project for WORLD magazine called <a href="http://worldontheroad.tumblr.com/">World On The Road</a>.</p>
<center>
<h3>Video - Part 1</h3>
<object width="400" height="300"><param name="allowfullscreen" value="true" /><param name="allowscriptaccess" value="always" /><param name="movie" value="http://vimeo.com/moogaloop.swf?clip_id=3887943&server=vimeo.com&show_title=1&show_byline=1&show_portrait=0&color=&fullscreen=1" /><embed src="http://vimeo.com/moogaloop.swf?clip_id=3887943&server=vimeo.com&show_title=1&show_byline=1&show_portrait=0&color=&fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="400" height="300"></embed></object>
<h3>Video - Part 2</h3>
<object width="400" height="300"><param name="allowfullscreen" value="true" /><param name="allowscriptaccess" value="always" /><param name="movie" value="http://vimeo.com/moogaloop.swf?clip_id=6049500&server=vimeo.com&show_title=1&show_byline=1&show_portrait=0&color=&fullscreen=1" /><embed src="http://vimeo.com/moogaloop.swf?clip_id=6049500&server=vimeo.com&show_title=1&show_byline=1&show_portrait=0&color=&fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="400" height="300"></embed></object>
<br />Video by <a href="http://vimeo.com/user1445026">James Allen Walker</a>.
</center>
<h3>Slides</h3>
<iframe src="http://www.slideshare.net/slideshow/embed_code/1195990" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px 1px 0; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/brian_dailey/nyphp-march-2009-presentation" title="NYPHP March 2009 Presentation" target="_blank">NYPHP March 2009 Presentation</a> </strong> from <strong><a href="http://www.slideshare.net/brian_dailey" target="_blank">brian_dailey</a></strong> </div>
<p>My dot files are available here:</p>
<ul>
<li><a href="http://dotfiles.org/~brian_dailey/.vimrc">Vim Configuration File</a> - There are a lot of additional plugins that I use, in addition to this.</li>
<li><a href="http://dotfiles.org/~brian_dailey/.bashrc">Bash Aliases & Functions</a></li>
<li><a href="http://dotfiles.org/~brian_dailey/.screenrc">GNU Screen Config</a></li>
</ul>
<p>Here is a copy of the NYPHP blurb:</p>
<blockquote>
<p>
It's March. Not much happens in March. Except, of course, for the March General Meeting, where we're pleased to have karate kid Brian Dailey show us how to slice through the original IDE - the command line. Even if you're an avid GUI user, these techniques are always useful - and great at impressing your friends - so join Brian and step up to get your own black belt.
</p><p>
The command line environment can be a powerful tool in a developer's toolkit. Learning to use it well can make you a more productive and more versatile developer.
</p><p>
This presentation is an introduction to the use of the shell as a development environment. The talk will go over the basics of shell navigation, including shortcuts that even seasoned CLI users may find useful.
</p><p>
The presentation will also briefly cover some of the most useful tools available to PHP developers: GNU screen for your multitasking needs, using vim as and IDE, and automated deployment with Phing.
</p>
</blockquote>
<h3>Links</h3>
<ul>
<li><a href="http://www.nyphp.org">New York PHP Users Group Community</a></li>
<li><a href="http://www.jamesallenwalker.com">James Allen Walker, Photographer</a></li>
</ul>
Open Coffee2008-05-01T00:00:00+00:00http://dailytechnology.net//random/2008/05/01/open-coffee<p><a href="http://entrepreneur.meetup.com/1124/">Open Coffee</a> was originally started in London, but Nicholas Butterworth, a friend of the original founder, started a group here in New York. It was a little slow gaining a foothold, but since a VC <a href="http://avc.blogs.com/a_vc/2008/04/meetups.html">blogged about it</a> it's exploded: mostly attracting folks looking for startup funding and seeking to "take it to the next level."</p>
<a href="http://flickr.com/photos/kungfoofool/2457325244/in/photostream/"><img src="http://farm4.static.flickr.com/3093/2457325244_7a30de7f47_m.jpg" border="0" alt="Open Coffee" align="right" style="margin-left: 10px; margin-bottom: 10px;"/></a>
<p>I go because the networking and discussion is great, and I think it's good to keep an eye on what the New York tech community is up to. What technologies are gaining in popularity? (Rails certainly appears to be on the upswing.) What new ideas are hitting the streets? How does a company go from a solo-start-up to an online phenomenon? There are a lot of interesting people at these meetups.</p>
<a href="http://flickr.com/photos/kungfoofool/2456498141/in/photostream/"><img src="http://farm4.static.flickr.com/3276/2456498141_6d6f2a56ba_m.jpg" border="0" alt="Open Coffee" align="left" style="margin-right: 10px; margin-bottom: 10px;"/></a>
<p>It does look like the venue is starting to get a little full. The format originally was meet and greet for a while, followed by introductions, and then coherent discussion. It's since devolved into multiple discussions at once, basically a free-for-all. I actually think that's a good thing since, to me, it's the networking that I come for, not necessarily discussing the macro-economy with fifteen other people. However, I'm sure Nick is looking to introduce some more structure and possibly a different venue.</p>
<p>I've seen a few other software developers (both solo guys and company fronts) at the meetings, and it's been interesting to see what sort of angle they take on things as well.</p></div>
Know Thy Code2008-03-16T00:00:00+00:00http://dailytechnology.net//random/2008/03/16/know-thy-code<p>When I first started programming in Java for a medical company (I was 17 at the time) I had very little training and unfortunately I learned a lot of bad habits that have taken a lot of study to identify and squelch.</p>
<p>Even what I learned in school, as others have often experienced, was far less useful than on-the-job experience and learning from other talented developers.</p>
<p>I once worked with another talented developer that didn’t use many of the tools that I’ve come to rely on: debugging aids, unit testing, high error/warning levels, etc. Yet this developer seemed to be able to catch nearly all bugs without any of these tools. At first I thought it was simply inhuman. How could someone catch everything on the first pass without any debugging tools or even compiler notices turned on? It wasn’t until I asked to sit down and have this developer walk me through his entire debugging process that light began to dawn upon why he seemed to be so thorough in reviewing his own code.</p>
<p>The primary difference was that I programmed with a web browser open while my colleague did not. My usual approach was to code a little bit, view the changes in the browser, make sure the incremental changes that I made worked (if unit testing is available, I’d run that, too) and then move on to the next incremental change. My colleague, on the other hand, would work on much larger incremental changes; he also spent much more time in the code itself checking for problems. By the time he was ready to fire up the browser (or in my case, a unit test) he was already confident that the code would work exactly as he expected it to.</p>
<p>I have recently been reading the book “<a href="http://www.amazon.com/Code-Complete-Practical-Handbook-Construction/dp/0735619670/">Code Complete</a>” by Steve McConnell on the advice of the programmer/blogger <a href="http://www.codinghorror.com/blog/">Jeff Atwood</a>. I have found it to be very useful and I have learned a lot from it, even though I’m only halfway through the book. One snippet from the book is quite relevant to this phenomenon that I witnessed.</p>
<blockquote>
<p><em>After</em> reviewing the routine, compile it. It might seem inefficient to wait this long to compile since the code was completed several pages ago. Admittedly, you might have saved some work by compiling the routine earlier and letting the computer check for undeclared variagles, [etc]. The main reason [for compiling later] is that when you compile new code, an internal stopwatch starts ticking. After the first compile, you step up the pressure. ‘I’ll get it right with just one more compile.’ The ‘Just One More Compile’ syndrome leads to hasty, error-prone changes that take more time in the long run. Avoid the rush to completion by not compiling until you’ve convinced yourself that the routine is right.” (Emphasis mine.)</p>
</blockquote>
<p>If you replace “compile” with “refresh the browser” you can apply the same idea to web application development. I realized after reading this that my colleague didn’t have superhuman powers: he had simply taught himself to thoroughly <em>know his own code</em> before loading it up and viewing it in a browser. I realized that I was guilty of “Just one more refresh” syndrome. I’m not discounting the power of unit testing or turning on all compiler warnings on a development machine. I still believe that’s vital to development in any environment. But change a line, refresh, fix the error that you saw, change another line… that’s a horrible way to develop software.</p>
<p>There are still many times in web development when firing up a web browser is still absolutely necessary, but I do believe that far too many web developers do not put enough emphasis on knowing and understanding the code before attempting to run it.</p>
<p>Here are some ideas that have helped me with knowing my code and catching problems before they ever appear in the browser.</p>
<ul>
<li>Close the browser. If possible, don’t open it until you have completely reviewed the code.</li>
<li>Review each routine (function) or class call and make sure it’s doing what you think it’s doing.</li>
<li>When you near finishing the feature, review the code and see if it can be refactored to make it more clear what is the code is doing (I’m a big believer in maximum code-clarity).</li>
<li>What does the code modify (for example, in the database)? How does it modify it?</li>
</ul>
<p>Of course there are many, many other things you can review (read Section 9.3 in “Code Complete” for some ideas) but the idea is to review, review, review, <em>before</em> opening the browser. After adopting this up-front investment in the code I have found that the number of errors in my code are much fewer and far between than they were.</p>
<p><strong>Edit:</strong> Hey, thanks for all the constructive criticism! One thing I should probably clarify. I never said that this idea applies to web <em>design</em>. Web design is (at least as I see it) defined differently from web (or software) development. Web design, as in HTML layout, CSS, images, what-have-you, <em>does</em> require that you view it in a browser early and often. It’s the nature of the game. What I’m referring to is completely on the backend (where, as some of you pointing out my “horrible font choices” probably already know, I spend 99.9% of my time). By the way, I will definitely take the recommendations on the serif font seriously.</p>
Marketing for Geeks2007-11-10T00:00:00+00:00http://dailytechnology.net//random/2007/11/10/marketing-for-geeks<p>One of the problems with what I do is that there are a <i>lot</i> of people out there who claim to be "web developers" but are really hack-jobs. I went to the <A href="https://www.paypal.com/us/developersday">Paypal Developer's Day</a> demonstration today and while a few of the 'developers' (I use the term very loosely) present seemed to be worth their salt the great majority of them really were hacks. The following conversation I overheard sums it up pretty well:</p>
<p>"Do you also work with PHP?"<br/>
"Oh, yes! Of course! But really I am waiting for the new PHP to come out. You know, PHP 5."</p>
<p>(For those of you not in the industry, PHP is a programming language, and version 5 has already been out for so long that version 4 has officially reached it's end-of-life.)</p>
<p>It's scary to think that <i>this</i> represents my local competition. These guys who really don't know diddly-squat about web security, web standards, or any of the above, and are willing to work for $9 or $10 an hour. Most small business owners are too technologically ignorant to filter this sort of thing out, and so they end up with a highly inferior (not to mention hackable, if you're gathering customer data!) product. They see that Developer X has "10 years of experience" (really consisting of Dreamweaver and FrontPage) and will work for 1/10th of what Developer Y asks for. The see the bottom line, the dollar, and not the product.</p>
<p>So what does that mean for me? Well, I already knew that the market for "web designers" (meaning anyone who can put together some sort of band website or small company website) is saturated everywhere. If someone posts a job like this on an internet forum like Craigslist, they get about 1000 responses; 99% of those responses are going to be tech guys in India, recruiters, or people out-of-state wanting to do work remotely. Obviously, with my New York rent, I can't compete with that. Nor do I wish to.</p>
<p>So when people ask "What do you do?" I'm faced with a dilemma. I cannot say, "I do web sites" because that puts me in the camp with the legions who wield the Dreamweaver sword. What I usually answer is that "I develop web applications." This better defines where my talents lie. I don't do websites for your local bands or your local dairy company. What I do is develop applications for business processes: inventory tracking, medical record systems, financial data, billing, etc. It's the same sort of thing that most desktop programmers did ten years ago, but now it's all moving to the web.</p>
<p>Contacts in these industries are far more difficult to make then your usual brochure-type website. They are, however, far more difficult to do and therefore tend to weed out the hacks that don't know what they're doing. I say "tend to" because, in truth, businesses often entrust their own internal processes to hacks and often suffer the inevitable meltdown later in the game. If you want stories about that, simply talk to any freelance developer or visit <a href="http://worsethanfailure.com">Worse Than Failure</a>. Whenever I see a business post an ad on Craigslist for an extensive application offering $10/hour, I am tempted to write down the business name so I can call them a year later and be the hero that fixes all of the stuff that the $10 hack left in horrible shape.</p>
<p>I believe that the market here for <i>good</i> developers is good. You have to be able to differentiate yourself from the crowd (which can be difficult when you're targeting people who know nothing about web security or software development) but if you can manage <i>that</i>, then I believe that the demand exists.</p>
Telecommuting Tips For Employers2007-11-07T00:00:00+00:00http://dailytechnology.net//random/2007/11/07/telecommuting-tips-for-employers<p>Many employers that used to be aghast at the idea of not being able to see their employees every morning are now a little more open to relaxing their grasp and allowing their employees to work from home. There have been many articles written about the benefits of telecommuting: employee happiness, less distractions, and other benefits. I've read several articles on how to avoid burnout (written from the perspective of the employee) and the other <a href="http://www.littlehart.net/atthekeyboard/2007/10/23/the-perils-of-telecommuting/">many perils and temptations of telecommuting</a>, but I've not read much in the way of advice for <i>employers</i>.</p>
<p>Having worked as a telecommuter in the past, I wish that some employers had been more aware of their impact on their distant employees. Here is some of my advice for employers thinking about hiring telecommuting developers.</p>
<ul>
<li><h3>Staying in touch is important.</h3>
<p>If an employee has ever worked from home or off-site, this is already obvious. We know that to keep our employer happy, we have to be available when you need us. It works both ways. If we have a question about something and it's impossible to get in touch with the employer (or worse, when we <i>do</i> get in touch, our employer seems to be annoyed at the interruption) then our job becomes more difficult. This is especially important when you're developing software. Questions may come up that need immediate and thorough answers.</p></li>
<li><h3>Keep the telecommuter in the decision-making loop.</h3>
<p>This especially applies to software developers, because it is important for us to feel we can influence what the company decides to implement. I've worked for companies in the past where the sales department was also the development department. <br/>
"We grow our market share by 2% if we add a feature that cooks the users breakfast!"<br/>
"Brilliant!" <br/>
It's generally a bad practice to promise a customer a feature you don't have yet (or, worse, sell software based on ephmereal promises of future development) but it's infinitely frustrating for a developer that wasn't there to mercifully kill the idea while it was still in it's infancy.
</li>
<li><h3>Have regular face-to-face gatherings.</h3>
<p>When a company is spread all over the country (or the globe) it's sometimes difficult to encourage the type of office comradery that you like to see in an office environment. If your departments never see each other, your developers will begin to associate your company with "the voice." It's like sitting in a room while a person behind a curtain dictates your never-ending "To Do" list. Workers become more and more disconnected over time until it hits critical mass and they either quit or are fired. We're human, and humans depend upon face-to-face interaction.</p>
</li>
<li><h3>Regular progress reports are not evil.</h3>
<p>If you are open to telecommuting, you have enough trust in your employees that you don't feel the need to stand over their shoulder and watch work they're working on. This is commendable. However, the extreme opposite is just as bad. If an employee doesn't feel that you're in the slightest bit interested in their work, then their work may suffer. I don't mean to suggest that you need to micro-manage from afar: just keep up to date on what they're doing and reward a job well done, just as you would if that person were working in your office.</p>
</li>
</ul>
How To Schedule Regular MySQL Backups In Windows2006-04-11T00:00:00+00:00http://dailytechnology.net//tutorial/2006/04/11/how-to-schedule-regular-mysql-backups-in-windows<p>Many web developers depend on that holy trifecta of PHP, MySQL, and Apache to develop their applications. Occasionally we are forced, whether due to work or convenience sake, to use Windows to host our applications. Some of us are already familiar with Linux cron jobs and shell commands (or maybe not) but scheduling backups in Windows is completely unknown to us.</p>
<h3>Some Initial Assumptions</h3>
<p>Here's the information you'll need to know before attempting to duplicate what I've done...</p>
<ul>
<li>The path to your mysql /bin/ directory (e.g., C:\mysql\).</li>
<li>Actually... that's it.... Easy, huh?</li>
</ul>
<h3>What We're Doing: In Summary</h3>
<p>We want to dump the file to a datestamped file. There are several ways to do this, but the easiest way is through a batch file. The command we'll be using is:</p>
<figure class="highlight"><pre><code class="language-bat" data-lang="bat">\path<span class="na">-to-mysql</span>\bin\mysqldump <span class="na">-u </span><span class="o">[</span><span class="kd">user</span><span class="o">]</span> <span class="na">-p</span><span class="o">[</span><span class="kd">password</span><span class="o">]</span> <span class="na">--result-file</span><span class="o">=</span><span class="s2">"\path\to\backup.</span><span class="vm">%DATE</span><span class="s2">:~0,3</span><span class="err">%</span><span class="s2">.sql"</span> <span class="o">[</span><span class="kd">database</span><span class="o">]</span></code></pre></figure>
<p>Just fill in the username, password, and database with your MySQL information.</p>
<h3>What Just Happened?</h3>
<p>You've dumped all of your MySQL data in <span class="note">[database]</span> into a file named backup.Mon.sql (assuming it's Monday). The <span class="note">%DATE:~0,3%</span> string basically runs the DOS "date" command and parses out the first three characters. If you'd rather see YYYYMMDD format, use <span class="note">%DATE:~10,4%%DATE:~7,2%%DATE:~4,2%</span>. The advantage (or disadvantage, depending on your needs) of using the week day is that you will eventually have only seven backup files... you can go back, at most, seven days. Eventually each day will be overwritten. Using YYYYMMDD will allow you to create an unlimited amount of backup files, and go back and recover to any date you'd like.</p>
<p>Take this command, and create a backup file (e.g., mysql-backup.bat). It can look like this...</p>
<figure class="highlight"><pre><code class="language-bat" data-lang="bat">@echo <span class="na">off</span>
<span class="nb">echo</span> <span class="kd">Running</span> <span class="kd">dump</span>...
<span class="kd">c</span>:\path<span class="na">-to-mysql</span>\bin\mysqldump <span class="na">-u</span><span class="o">[</span><span class="kd">user</span><span class="o">]</span> <span class="na">-p</span><span class="o">[</span><span class="kd">password</span><span class="o">]</span> <span class="na">--result-file</span><span class="o">=</span><span class="s2">"c:\path\backup.</span><span class="vm">%DATE</span><span class="s2">:~0,3</span><span class="err">%</span><span class="s2">.sql"</span> <span class="o">[</span><span class="kd">database</span><span class="o">]</span>
<span class="nb">echo</span> <span class="kd">Done</span><span class="err">!</span></code></pre></figure>
<p>Now you can run mysql-backup.bat from a command prompt (hit Start->Run and type 'cmd'). It it works, you should see a success message and a file should be generated in the directory you specified. If not, make sure you've got the correct username, password, and database. Then try it again.</p>
<h3>Automating It All</h3>
<p>Linux has cron, Windows has <span class="note">at</span>. The at command schedules commands to run as often as you'd like. In this case, we're going to run the backup one day a week at 2 AM.</p>
<p>Open up the command prompt and type the following:</p>
<figure class="highlight"><pre><code class="language-bat" data-lang="bat"><span class="nb">at</span> <span class="m">02</span>:00 <span class="na">/every</span><span class="nl">:M</span><span class="o">,</span><span class="kd">T</span><span class="o">,</span><span class="kd">W</span><span class="o">,</span><span class="kd">Th</span><span class="o">,</span><span class="kd">F</span><span class="o">,</span><span class="kd">S</span><span class="o">,</span><span class="kd">Su</span> <span class="kd">c</span>:\path\mysql<span class="na">-backup</span>.bat</code></pre></figure>
<p>Once you run this, it should return "Added a new job with job ID = X"</p>
<p>Type <span class="note">at</span> to make sure that it's in the schedule.</p>
<h3>Congratulations!</h3>
<p>That's all you need to do! If you've set everything up correctly you should now be running nightly MySQL backups.</p>