Caffeinated Ideas

This Week in Cats: EitherT

Fri, 25 Jun 2021 00:00:00 +0000

Exceptions are the antithesis of functional programming since they break referential transparency, the code after resolving the exception cannot be substituted for the code prior to the exception. To avoid exceptions but still communicate errors to callers, we end up using the Either type throughly through the code. We also want our code to be async and so we also use the Future type in the result which leads to many methods like:

This Week in Cats: Eval

Wed, 05 May 2021 00:00:00 +0000

One of the big patterns in software is the idea of evaluating some piece of logic once. Normally this is done to obtain a value from a function or method call that takes some time to compute and Scala proposes using lazy val for this purpose.

@ def squareSlowly(i: Int): Int = {
 Thread.sleep(10000)
 i * i
}
defined function squareSlowly

@ squareSlowly(10)
// ... for 10 seconds ...
res1: Int = 100

@ lazy val tenSq = squareSlowly(10)
tenSq: Int = <lazy> // notice the immediate return

@ tenSq
// ... for some time up to 10 seconds ...
res3: Int = 100

@ tenSq
res4: Int = 100 // notice the immediate return

The problem here is that tenSq is a variable and not a function call. If you wanted to pass tenSq into methods or functions, you would need to use the call by name argument:

This Week in Cats: Kleisli

Fri, 09 Apr 2021 00:00:00 +0000

Kleisli. It’s got a funny name but it’s actually a simple concept. Suppose you have two functions that you want to execute together:

val computeSquare(i: Int): Int = i * i
val formatResponse(i: Int): String = s"Your answer is $i"

// One way of doing it is
val sqrAns = computeSquare(3)
val strAns = formatResponse(sqrAns) // strAns = "Your answer is 9"

// or avoiding naming variable
val strAns = formatResponse(computeSquare(3)). // strAns = "Your answer is 9"

Instead of needing to call these two functions over and over, either as individual lines or nested, you can easily create a new function from the two that always does both using function composition:

This Week in Cats: Alternative

Fri, 26 Mar 2021 00:00:00 +0000

Let’s say I have two methods that return Either and I just want to take the first successful result, only defaulting to the error return if both fail. For example, each method is going to look at different fields of a request body and if the required fields are present, build an object.

trait Credential
case class CredType1(username: String, password: String) extends Credential
case class CredType2(token: String) extends Credential

case class ReqBody(username: Option[String], password: Option[String], token: Option[String])

def maybeCredType1(req: ReqBody): Either[Throwable, Credential] =
 Applicative[Option]
 .map2(req.username, req.password)(CredType1)
 .map(Right(_))
 .getOrElse(Left(new Exception("Cred1 requires both a username and password")))

def maybeCredType2(req: ReqBody): Either[Throwable, Credential] =
 req.token
 .map(value => Right(CredType2(value)))
 .getOrElse(Left(new Exception("Cred2 requires a token")))

I could write some additional logic that calls the correct method based on the values in ReqBody but then I’d be duplicating the logic about which fields are required for which implementation of Credential. This is where we want to call both and take the first success. There are a number of ways I could implement that, pattern matching, for comprehension, nested maps, but as I suspected this is a solved problem. Cats defines the <+> operator to implement this kind of alternative matching behavior.

This Week in Cats

Thu, 25 Mar 2021 00:00:00 +0000

As I mentioned last April, I ended up switching jobs and found myself in a Scala shop that embraced the Typelevel functional programming style. As I worked on tickets and shippped features, I started to deep dive into the cats library. Many of the concepts made my life easier and so I sarted to share them internally in a learning slack channel.

It then dawned on me that they might also be useful for others. As time permits, I’m going to clean up the slack posts and start a new series - This Week In Cats. It’s probably a mistake to committing to a weekly post but that’s the name of the series in slack.

2020 Q2 Tech Radar Update

Mon, 29 Jun 2020 00:00:00 +0000

With 2020 closing in on the half way point, it’s time to reflect back on the goals I had set this year and see how they’re holding up.

Reviewing the Q2 Updated Plan

Going into Q2, I made a lot of adjustments to my tech radar goals when I moved to Rally. The move to Rally was a very positive change but required ramping up on Scala - something I did not anticipate at the beginning of the year. So, how did I do on my new list?

Updating main git branch name

Tue, 16 Jun 2020 00:00:00 +0000

There’s a lot of terminology we use without thinking about the meaning of those words. When I started tinkering around with the insides of the IBM PC and configuring hard drives, it was common to see references to the master and the slave drive in the BIOS and on the jumper settings. Those same terms were used in textbooks and lectures about database systems and highly available system designs in the 90s.

Scala's foldLeft Operator Demystified

Thu, 07 May 2020 00:00:00 +0000

Back in 2012 I wrote a blog entry, Scala Syntactical Heartburn, in which I said

The first approach, using the /: operator, doesn’t make sense. I’m not sure how I get foldLeft, inject, or whatever else you want to use to describe the operation out of this symbol.

Well, that changed this week while reading Functional Programming in Scala. In chapter 3, there is a discussion on how to create an immutable, functional List type and doing interesting operations on the list, like summing the list. Enter foldLeft and foldRight.

Type Hint ProcessBuilder Calls

Fri, 24 Apr 2020 00:00:00 +0000

One of the many things that OpsCenter had to do was spawn new processes with custom environment variables using the environment() method on the ProcessBuilder class. The method, which embraces mutable state, returns a Map<String, String> that you need to update with whatever new values you want to pass through to the new process.

(defn start-application []
 (let [cmd (into-array String ["java" "-jar" "App.jar"])
 proc-builder (ProcessBuilder. cmd)
 env (.environment proc-builder)]
 (.put env "ENV_NAME" "dev")
 (.start proc-builder)))

The start-application function spawns a new Java application while setting the ENV_NAME parameter for the local machine and returns the new Process object.

2020 Q1 Tech Radar Update

Wed, 08 Apr 2020 00:00:00 +0000

Hard to believe but we’re already three months into 2020 and it’s time for a quick retrospective on my 2020 Tech Radar goals.

Exploring: TLA+
No significant progress to update on this. I’ve read through the first chapter of Practical TLA+ but didn’t go beyond that.
Deep Dive: Rust
Started reading The Rust Programming Language book. I’m two chapters in and so far it’s still pretty standard language stuff.
Deep Dive: Reason
No progress.

Exceptions Ruin Code Flow

Wed, 19 Feb 2020 00:00:00 +0000

We often insist that anyone writing code understand and handle errors using exceptions. By their definition, exceptions force you to break your thought process on the present line of code and require that you immediately start to think about how to handle the error condition. In some cases, like named exceptions in Java, there is no agency. You must either deal with the exception or just declare it as a possible error to your code, pushing the concern to some other place in the code.

My 2020 Tech Radar

Wed, 15 Jan 2020 00:00:00 +0000

Every year, I like to set goals for myself to help me focus on new things that I want to learn or adopt throughout the year. I typically model my goals on a layout similar to the ThoughWorks Radar that is published every quarter. The three main categories that I’ve used in my tech radar are:

Exploring
Topics listed in the “exploring” section are items that I’ve run into either in conference talks or papers and want to know more about them. This is usually limited to either reading a book, paper or watching conference talks on the topic. Cursory knowledge is the goal with these topics.

Migrating to Clojure Deps

Thu, 26 Dec 2019 00:00:00 +0000

Over the last couple of months, I’ve worked on migrating my static site generator code from boot to Clojure’s new deps tooling. Deps is fairly minimal in what it provides and that turns out to be exactly what I need.

Background

Back in 2017, I outlined my motivation for building my own static site generator. At the time, I selected boot as the platform to build off of. Boot abstracted a lot of concepts common to build systems and had a large plugin ecosystem that I would leverage for various aspects - like syncing files to AWS S3. This allowed me to focus my efforts on getting the representation of the site content correct and building out the core functionality.

Code Needs Maintenance

Mon, 05 Aug 2019 00:00:00 +0000

This weekend I had to replace the top deck boards on the steps leading to our front porch. The work seemed easy enough - buy some wood, rough cut to length, unscrew the old boards and screw in the new boards. It’s only eight steps so it shouldn’t take more than a day if I’m slow. After spending a whole day on the project, I only managed to replace three out of the eight steps.

Building a Clojure to Jython Interop

Tue, 16 Jul 2019 00:00:00 +0000

For the last four years, I’ve been working at Datastax on the OpsCenter product team. OpsCenter itself was originally built with Twisted on python and leveraged small agents, written in Clojure, to interact with every Cassandra node instance. Shortly after I started, the core team decided to port the python component to Jython, a JVM based Python implementation. This isn’t a secret or propitiatory information since our architect, Nick Bailey, gave a talk at Clojure/conj 2016 about our efforts.

Problems with flatten on Java types

Wed, 23 Jan 2019 00:00:00 +0000

The bug

Our automated functional tests caught an exception in a block of functionality that needs to read a sequence of strings from a deeply nested YAML file structure.

(defn load-yaml-contents
 []
 ;; This is a mocked set of data from parsing the YAML file.
 (map identity [["one" "two" "three"] ["four" "five"]]))
 
(-> (load-yaml-contents)
 flatten
 (reduce concat []))
=> (\o \n \e \t \w \o \t \h \r \e \e \f \o \u \r \f \i \v \e)

After flatten we would have a list of strings but then when that sequence is reduced using concat, each string value is treated as a sequence of characters and each character is appended to the vector. This is not the result that we want. I then had to question if this code ever worked. There was a unit test around this behavior that asserted the result of this code produced a sequence of strings. This unit test was last touched when the code shipped and has been running successfully in nightly builds. I was perplexed as to why the test passed but the code failed in an end to end test.

Why Learn a Lisp

Fri, 22 Dec 2017 00:00:00 +0000

I’ve been fortunate enough since April 2015 to work in a position where I was expected to write and support Clojure code as one of my primary responsibilities. Over that time, Clojure has changed me as a programmer. It wasn’t until I listened to episode 68 of the Functional Geekery podcast that I really reflected on how learning a lisp language has changed me.

Many people who work with a lisp language tend to speak of this transcending experience and enlightenment that comes with learning a lisp. After hearing these kinds of vague experience reports and all sorts of success stories with Clojure, I was determined to become a Clojure developer. I pushed Clojure at QThru, tried to introduce it at Medio and even selected Clojure as a key technology at jetway.io. Through all of this, I couldn’t tell you an exact reason other than elegance and beauty - neither of which solved a business problem for me.

What herd behavior can teach us about how we estimate work

Fri, 15 Dec 2017 00:00:00 +0000

This year I managed to finally read Nate Silver’s book “The Signal and the Noise: Why So Many Predictions Fail – but Some Don’t”. The concepts in the book will feel similar to any developer who’s read up on or attended any agile session on estimating. In Chapter 11, Silver turns to looking at stock markets and why the modern market behavior has changed when compared to the market behavior of the past.

Rebooting QThru

Tue, 28 Nov 2017 00:00:00 +0000

“Our greatest weakness lies in giving up. The most certain way to succeed is always to try just one more time.”

– Thomas Edison

Shortly after Aaron informed everyone on July 8th, 2013 that QThru was out of money, I met with one of my coworkers. We talked a lot about projects that were just left unfinished and the potential we saw in the product. The most interesting question was whether I would return to QThru if Aaron was removed and a new CEO was put in place. In fact, a new CEO with experience selling into retail markets was just what QThru needed as Aaron was unable to book any deals since we got funding. With that single answer, I started helping produce a plan to adjust course with QThru assuming that the company could be taken over.

Benchmarking Clojure Performance with a byte counting InputStream

Wed, 25 Oct 2017 00:00:00 +0000

OpsCenter’s goal is to deliver operational simplicity around deploying and managing a DSE (e.g. Cassandra) cluster. One core feature of this offering is backup and restore functionality that snapshots the individual nodes and then copies that data to either AWS S3 or a local file system mount. The component that performs this task, the DataStax agent, simply views all the data as Java streams instead of individual files on the disk.

Modeling Sales Tax Rates

Sat, 23 Sep 2017 00:00:00 +0000

“The hardest thing in the world to understand is the income tax.”

– Albert Einstein

Consumers don’t often think about the legal requirement that a sales tax must be paid by them for items they purchased based on where the transaction took place. Retailers with physical locations very often collect and submit that tax on behalf of the consumer to make the transaction easier. This means that any point of sale needs to have support built in to compute and add the sales tax on to the cart.

QThru: Data Model

Fri, 22 Sep 2017 00:00:00 +0000

“Data is a precious thing and will last longer than the systems themselves.”

– Tim Berners-Lee

The mobile platform we developed at QThru started as a simple mobile application with a ReST API to perform CRUD operations. The types of data that we stored was one of the more valuable assets that our platform was generating. Over the 2 years of operation, the data model was transformed and enriched as we better understood the domain. In this post, I’m going to point out some of the decisions we made about our data model and whether they were helpful or not.

Building My Own Static Site Generator

Fri, 25 Aug 2017 00:00:00 +0000

I’ve been pretty quiet on this blog for the last year as I worked on my own static site generator. This wasn’t my original intention I wouldn’t recommend building from scratch for people who just want to publish content. This story starts back in 2015 when my site was built using jekyll and hosted on S3.

Toward a Clojure solution

Sometime around 2012 or 2013, I started learning more about static site hosting using AWS S3. The cost would be pennies compared to the $10/mo I was paying for hosting a WordPress blog. That was a significant savings for a site that I assume gets very little traffic. I started the process of exporting the old posts from WordPress and building out a site with jekyll, the most popular static site generator at the time. I was a happy jekyll user until I read The Pragmatic Programmer.

The Five W's of Engineering Leadership

Wed, 09 Aug 2017 00:00:00 +0000

In the last couple of months, I’ve had some interesting conversations with my employer about my career. Frankly, I’m not sure what I want to do next. One thing that has come out of these conversations is a simple abstraction I’ve used to think about different positions and what they do. I’ve based it on the Five W’s.

Who (individual assignments) - Team Lead
What - Product Management
When - Project Management
Where (which team) - Engineering Manager
Why - CTO
How - Architect, Tech Lead

There are plenty of corner cases but at least having some high level abstraction has given me some guidance.

QThru: Warp Pipe POS Integration

Fri, 19 Aug 2016 00:00:00 +0000

A lot of the Google inventions came from engineers just screwing around with ideas. And then management would see them, and we’d say, ‘Boy, that’s interesting. Let’s add some more engineers.’

-Eric Schmidt

One of the largest challenges the engineering team faced at QThru was the initial setup with a retail location. Our proof of concept code required a CSV file with product and pricing data that we would then import into our database. We thought that the CSV format would be consistent between point of sale (POS) systems and that we could just use a copy of that data file. This was not true. We also assumed that the pricing managers and staff at the grocery stores would comfortable setting up the import program. This was also not true.

QThru: The Platform Architecture

Tue, 28 Jun 2016 00:00:00 +0000

Good judgment comes from experience, and experience comes from bad judgment.

– Frederick P. Brooks

Now that the NDAs have expired and QThru has officially folded, I think it’s important to publish what we learned from building a mobile commerce company by a set of engineers that only had experience in the email and message space. What we designed had some good parts but also some failures.

This is the third in a series of retrospective thoughts on QThru, a mobile self checkout startup that went bust. See my “QThru Series” post for a list of all the topics.

QThru: The Business Model

Thu, 16 Jun 2016 00:00:00 +0000

I’m talking about liquid. Rich enough to have your own jet. Rich enough not to waste time. Fifty, a hundred million dollars, buddy. A player. Or nothing.

– Gordon Gekko

This is the second in a series of retrospective thoughts on QThru, a mobile self checkout startup that went bust. See my “QThru Series” post for a list of all the topics.

In 2009, the smartphone app market was on fire. Apple’s decision to release a native app SDK that developers actually wanted to use unleashed the creativity of the world and new businesses started to emerge. In particular, startups were looking at how an Internet connected computer could be used to process payments. In February that year, Square launched their mobile payment platform for iOS.

The QThru Series

Fri, 03 Jun 2016 00:00:00 +0000

One who fears failure limits his activities. Failure is only the opportunity to more intelligently begin again.

– Henry Ford

In April 2012, I ended up meeting with Aaron Roberts for lunch. I had just left BlackBerry for another position that didn’t quite live up to my expectations. A few days prior, Aaron and I talked about a startup he was building called QThru. Before we got too far into lunch, I flat out asked him if he had room for another developer and we talked about how to proceed. After a quick meeting with the existing dev team, I walked away from a paying job with benefits to become the first, and currently unpaid, full time employee for QThru.

Another argument for s-expressions

Thu, 22 Oct 2015 00:00:00 +0000

I found myself back in our Python code base today working with a set of nested dictionary objects. As I set out to look for the equivalent of Clojure’s get-in function, I ended up at a nifty solution using reduce. Inspired, I ended up with the following function in our Util module:

def get_in(d, keys, notfound=None):
 """Python implementation of Clojure's get-in function."""
 try:
 return reduce(dict.__getitem__, keys, d)
 except TypeError, exceptions.KeyError:
 return notfound

Of course this failed the unit test that I wrote:

stat command

Tue, 01 Sep 2015 00:00:00 +0000

I was working on a shell script today that required determining the user that owned a particular directory. A quick search online led me to this Stack Overflow post. This led me to the following quick command that I could use in a script

CASSANDRA_USER=ls -l /var/lib | grep cassandra | awk '{print $3}'
USER=${CASSANDRA_USER|-"cassandra"}

The second line sets the var USER to either the value in CASSANDRA_USER or to the default string “cassandra” if the previous command did not result in a value. The grep was the weak point and it actually started to fail when another component added a directory to /var/lib that was owned by a user named cassandra.

Podcasts About Development

Mon, 17 Aug 2015 00:00:00 +0000

When I used to commute to the office in Seattle, I would listen to a number of development related podcasts. They were a great way to get exposed to new trends as well as wake up my brain before getting to the office. This is a list of a few of my favorite podcasts that have withstood the test of time. Let’s jump right into the list.

The Cognicast

Primarily focused on Clojure and ClojureScript development, this podcast always seems to come up with interesting topics to talk about. Previous shows have included deep dives into Om, property based testing, writing code without a type system, data structures, dyslexia and more. It’s roughly an hour long, has a consistent release schedule and a wide range of guests.

Why You Need Macros

Thu, 06 Aug 2015 00:00:00 +0000

In 2013, I started looking into a Elixir, a new language that looks like Ruby and runs on BEAM. One of the more interesting parts of the language is the early development of the [macro](https://en.wikipedia.org/wiki/Macro_(computer_science) system, an idea borrowed from Lisp. With macros, the Elixir developers were able to iterate and provide a wide array of control structures without the need to bloat the core language.

While many developers look at macros as things to avoid, they are essential in building languages today. Without macros, a language needs to produce a big design first in order to ensure that the new feature fits within the large language ecosystem. C# and Java are prime examples of slowly evolving features that need to be blessed before they can be used. This leads to long release cycles where the cost of failure is high. Yet, the Clojure community was able to introduce core.async without requiring any impact to the core language library. At the heart of core.async is a complex state machine abstracted through a series of macros.

Great developers build tools

Fri, 03 Apr 2015 00:00:00 +0000

It was important, his father said, to craft the backs of cabinets and fences properly, even though they were hidden. “He loved doing things right. He even cared about the look of the parts you couldn’t see.”

– Walter Isaacson. “Steve Jobs.” iBooks. https://itun.es/us/QyFUz.l

Today I had to investigate a ticket that failed the end to end verification step. The comment in the ticket contained a good amount of details about the environment that the test was run in, what was performed and a little bit about the failure. It was more details than a customer would provide but didn’t include log files, memory dumps or live debugging.

Superfish Highlights the SSL Problem

Fri, 06 Mar 2015 00:00:00 +0000

It’s only been two weeks since news of Lenovo bundling Superfish on their laptops was brought to the public’s attention. While the focus has been on who at Lenovo knew about the bundle, few people are really talking about what it means to the security of the Internet. Developers used to just enable SSL on their services to check off their security requirements. Developers now need to think more about security and should pay attention to the new research underway.

Is node.js turning into the new Java?

Wed, 18 Feb 2015 00:00:00 +0000

It’s been quite a ride for for node.js. A group of core developers secured a nice series A round to start npm, Inc., bringing a corporate steward to the growing repository site. Success stories on node.js in various companies seem common place with new stories weekly. Recently, NodeSource closed their series A round to bring node.js to enterprise customers. All this excitement for node.js and JavaScript was succinctly summed up in 140 characters:

Brendan Eich: JavaScript's destiny is to fulfill Java's promise http://t.co/KFM59kIz20

Java 'synchronized' in Python

Fri, 12 Dec 2014 00:00:00 +0000

A recent student posed the question, “Is there a way to lock execution of an entire method in Python using Lock instead of writing out a with-block?” My answer started with all the good reasons why we wouldn’t want to hide the locking behavior in magic, why adding it to all methods on all objects in python would be bad for performance and testing and then to argue that the current syntax based on the with keyword isn’t that painful. Then I tried to see if I could replicate the problem using some python magic.

How Certain Are You?

Fri, 03 Oct 2014 00:00:00 +0000

At some point in our lives, we’re given this misconception that with effort and training we can have some level of certainty about the future. On a small enough scale with very fixed boundaries, I suppose that’s true - it’s probably pretty certain that I’ll take another breathe of air. Even then there is still a small level of uncertainty that I won’t and the outcome of that action will make for a pretty wild day. The harsh reality is that nothing in life is certain.

Breaking Statically Typed Languages

Thu, 14 Aug 2014 00:00:00 +0000

Every time I tell my fellow Java developers that I enjoy writing code in python or node.js or clojure, I’m very often dismissed with a set of reasons but the one I always enjoy is:

Java is statically typed so the compiler checks to make sure I don’t make mistakes.

Yet many Java developers don’t use the type system.

Well, they really do use the type system but just as a way of declaring that everything in the system is some built in type - typically integers, strings or booleans.

JavaScript Dependency Injection Pattern

Mon, 17 Feb 2014 00:00:00 +0000

Over the course of the last month, I’ve been working on rebuilding a JavaScript SDK for my employer. Once of the requirements was that all of the code needs to ship with an automated test suite that capable of testing a majority of the basic functionality. Throughout the process, I’ve relied a lot on dependency injection in order to provide hooks to mock or stub out functionality in a fairly straight forward manner.

TIL Culture Matters

Mon, 13 Jan 2014 00:00:00 +0000

For me, 2013 was a year of many lessons learned out of the unfortunate bankruptcy of QThru. It just seemed right to document all of the things I’ve learned through the process. This is the first post in my TIL (things I learned) series, focusing on things that engineers don’t think about while working at a startup company. The first lesson is that culture really does matter.

If you’re an engineer at heart and you spend your time building things, you are probably inclined to zone out when people start talking about mission statements, values, review process and org charts - the culture. In fact, one of the appealing upshots of working at a startup is that the management team is thin and isn’t shoving these concepts down your throat at monthly or quarterly meetings. At first, it can seem like a breathe of fresh air that lets you get down to the business of building the product.

Taming node.js with Clojurescript

Thu, 29 Aug 2013 00:00:00 +0000

In my previous post, I outlined a new, lighter weight approach to building web applications using Clojurescript and node.js as the runtime environment for the backend. The nice part of Clojurescript is that it doesn’t try to abstract away the host environment so all those useful node.js packages can be reused without the need to wait for someone to rewrite them in Clojurescript.

To start, we need to pick a framework for our web development. My personal preference is restify since it comes out of the box with support for API versioning, a static file server and handles a fair number of ReST conventions without any configuration. The only thing that it lacks is built in support for a page templating library, but this isn’t important for our application. All of the pages in our app will be static HTML files and we’ll use client side Clojurescript to load data and respond to the user actions.

Webapps with Clojurescript

Thu, 15 Aug 2013 00:00:00 +0000

In the late 90s and early 2000s, Java and C# sparked a fundamental change in how software is developed. With the introduction of fast garbage collection and VMs executing bytecode, developers stopped worrying about mundane issues and were free to tackle much more complex problems. This has lead to large system components like hadoop, cassandra and solr, which give the world a huge base to build the future on. Yet, software development now has new problems in terms of concurrency and correctness that Java and C# fail to provide ideal solutions to.

Smarter Architectures with NoSQL

Sun, 16 Jun 2013 00:00:00 +0000

The pace of software development and the demands for teams to produce features has been steadily increasing. This increasing pace has driven the industry to innovate how we create software, like automating our tests and deployments. Another tools that can help development teams deliver features faster are NoSQL databases.

It’s unfortunate that a lot of conference talks about NoSQL end up focusing on the technical scalability of the products. Most shops aren’t going to need to worry about building web scale solutions initially, and may never need to build to web scale standards. This doesn’t mean that we should then write off NoSQL databases. In addition to handling higher loads, many NoSQL databases already implement a lot of functionality modern apps use.

Autowiring Akka Actors with Spring

Mon, 18 Feb 2013 00:00:00 +0000

We consume a lot of data in various formats to deliver our mobile shopping experience and we need to process these data feeds quickly so we can offer up-to-date pricing and inventory within our application. This means that we need to avoid blocking operations as much as possible during our calculations and run multiple elements through our import process concurrently. Instead of rewriting tons of basic threading code, I wanted to leverage Akka’s actor based model so we could focus our code on the actual business logic of importing the data and less on threading.

Hibernate Tips

Wed, 12 Dec 2012 00:00:00 +0000

I’ve decided to assemble a list of some cool features I found in the last week working with Hibernate. If you’re going to use an ORM, you might as well learn to use it effectively.

Tip 1: Scan for Entity Classes

If you’re using Spring ORM and Hibernate like we are at QThru, you’ve probably started off with a session factory that listed each of the entity objects in the application context XML file. To avoid repeating what you’ve already declared with annotations, use the packagesToScan property to have Hibernate discover those entity objects for you:

My Journey with Seven Languages

Mon, 17 Sep 2012 00:00:00 +0000

Being nearly a year behind on my reading list, I finally broke out my copy of Bruce Tate’s Seven Languages in Seven Weeks book for a little mind expanding exercise. At first I was a little disappointed that I’ve already worked with most of the languages back from my days in the CS program at UWM. Even so, I decided to keep my mind open and I’m glad that I did.

Documenting ReST - Part 2

Fri, 07 Sep 2012 00:00:00 +0000

In part 1 of this post, I laid out the problem facing us and how I was going to tackle documenting our ReST API. After learning more about the JavaDoc tool, extending it seemed like the logical solution that didn’t reinvent the wheel. To make this whole project more interesting, I also decided that I’d try my hand with Clojure. It’s a language I’ve been meaning to play with for awhile and this seemed like a small enough task with some interesting meaty bits, such as exposing Java classes to other tools.

Documenting ReST

Mon, 20 Aug 2012 00:00:00 +0000

As a startup, we only have a limited number of resources and a huge product backlog to get through (who doesn’t). To accelerate delivery of a bunch of features, we had to de-agile our team and outsource parts of the development efforts. While we decided to outsource important items, we don’t necessarily want to loose control of the cloud platform. This has lead to a silo between our backend service developers and the new outsourced group and some interesting communication concerns. The largest is, how do we communicate our API to those using the API?

Apple's Mountain Lion Up-to-Date Program

Wed, 25 Jul 2012 00:00:00 +0000

This year I made the decision to buy a new laptop. My MacBook Core Duo just was struggling to keep up with all the programs that I use on a daily basis for development. Running VMs with Linux and Windows didn’t help so I jumped on board the new 15" MacBook with Retina display because I could get 16GB of RAM in it. The added bonus was a free copy of Mountain Lion when it was released. I’m still waiting on my download code from Apple’s up-to-date program and that’s a bit surprising.

Scala Syntactical Heartburn

Thu, 12 Jul 2012 00:00:00 +0000

This morning I was working through the Scala chapter from “Seven Languages in Seven Weeks” by Bruce Tate and discovered there are three ways to apply a cumulative operation across all items in a list using foldLeft().

We start with the syntactical sugar version first:

val list = List(1, 2, 3)
val sum = (0 /: list) { (sum, i) => sum + i }

The /: operator is just a shorthand notation for the function foldLeft and could be written out in long form as

RIP LandlordCompanion.com

Thu, 28 Jun 2012 00:00:00 +0000

I ran across this article about Rentobo while glancing through TechCrunch the other day. This sounds a lot like the idea I had for a site I was working on from 2005-2010 called LandlordCompanion.com, which failed to launch after many years of trying. Instead of yet another blog posting about a success story, I figured I would write about my failure. Hopefully someone else can learn from my mistakes.

In 2005, I was in the search for a new place to rent with my girlfriend. We wanted something a little bit more like a house and turned to Craigslist to find rentals. The competition was pretty heavy, often with us sitting in the kitchen filling out applications with other prospective tenants. Much of the process was manual and often the landlords were individual people with day jobs. That made the whole process of turning in application fees more difficult. I felt there had to be a better way and why not use the Internet to do it.

ORM Rant, Continued

Thu, 24 May 2012 00:00:00 +0000

Back in January, I wrote an entry about not being a fan of ORM frameworks in software development. At the time, we were struggling to increase the performance of the native mail, calendar and contact clients on the PlayBook. Our use of SQLAlchemy hindered us by trying to abstract out database calls, sometimes in ways that made the performance worse. Truly getting the performance numbers we wanted often involved solving the data access problem and then spending just as much time to figure out how to solve the problem again with SQLAlchemy. It would have been easier to just write the SQL by hand and only used on level of abstraction.

Structuring and Naming Unit Tests

Thu, 29 Mar 2012 00:00:00 +0000

Unit testing and automated unit testing is part of all conversations around software development these days. It’s used by the agile groups to point out how software gets developed without huge, monolithic testing ground and by quality groups to prove that early testing reduces the overall cost on an organization. Despite the topic being so wildly discussed and required in modern development, we don’t really talk a lot about the nuts and bolts of development, specifically how to structure and name the test code that we write.

Rumination on Social Forces

Fri, 27 Jan 2012 00:00:00 +0000

Every morning, I start my day getting caught up on the tech news and latest happenings in the world. All of these products continue to tout their social capabilities or how they perform better with a social effect. Everywhere you turn, it seems that tech companies are baking a social aspect into their products. Is this always a good idea and should the inclusion of these social features help? That depends entirely on the social forces involved.

Yup, I'm Still Not An ORM Fan

Fri, 13 Jan 2012 00:00:00 +0000

Object-Relational Mappers (ORM) frameworks have taken off in the agile development community in the last decade under the prose that relational databases are too complex for developers to deal with and we should instead deal with objects in our code. After working with a few different ORM frameworks, I’m still not sold. All the perceived productivity gains are quickly a wash once the team starts to encounter the common ORM problems. I see three large problems cropping up with code bases that incorporate an ORM framework:

Task Queues Over Threads

Fri, 06 Jan 2012 00:00:00 +0000

I hate threads.

More accurately, I hate the abstraction the industry has settled on for trying to do work in parallel. When the computer scientists sat down and created the first threading API, they damned all future computer scientists to the wonderful joy of debugging shared memory states and complex issues revolving around an API that you can’t help but shoot yourself in the foot with. For instance, most systems let you start a thread but then leave actually halting a thread to be this complex, tangled mess of state signaling, join methods (which might not actually join in the end) and a boat load of blocking issues.

Why Blog?

Mon, 12 Sep 2011 00:00:00 +0000

I recently started reading the book Coders At Work: Reflections on the Craft of Programming and this snippet between Peter Seibel and Jamie Zawinski jumped out at me.

Seibel: Do you find that programming and writing are similar intellectual exercises?

Zawinski: In some ways, yeah. Programming is obviously much more rigid. But as far as the overall ability to express a thought, they’re very similar. Not rambling, having an idea in your head of what you’re trying to say, and then being concise about it. I think that kind of thinking is the overlap between programming and writing prose.

What Navy SEALs Can Teach Us About Software Development

Mon, 09 May 2011 00:00:00 +0000

I was having a conversation with a co-worker last week about various bad scrum practices that seem to creep into acceptance (often times called scrum-but). We started to think about why we’ve seen scrum teams with these problems and what the solution is. That’s when I asked, “how come the Navy SEALs don’t seem to have as many problems even though they face great uncertainty and challenges?” After thinking about it, I came up with three key lessons that we could learn from the SEALs.

Stone Age Project Teams?

Wed, 20 Apr 2011 00:00:00 +0000

Like everyone else in the industry, our team has embarked on the adventure to adopt a variety of agile practices over the last couple of years. First, we picked up scrum and started to develop features in iterations (despite our still waterfall like release model). After a few projects with scrum, many members of the team took up test driven development. After a lot of stumbling, we started to wrap our minds around TDD and writing good unit tests. Now, we have some team members that want to adopt pair programming. Yes, I’ll admit that I’m not a huge fan of pair programming but I tried a couple of sessions with a few different people. The whole time I couldn’t get over the biggest problem that I’ve found with pairing and that’s the need for a team that works the same hours and is preferably co-located. In today’s world of globally disperse teams, it seems wrong to sell people on the need to adopt a coding practice that requires a team from the 20th century.

State of the Union: Application Logging

Thu, 20 Jan 2011 00:00:00 +0000

Over the last week, I’ve been asked to help investigate some issues with a new obfuscation process that our builds will now go through before deployment (we can save the rants about obfuscating code for some other time). Per the norm for this activity, the test group ran into issues in a section of code and needed help investigating. As I dug into the issue, I turned to the log files and found that they were absolutely worthless. While we suspected that the obfuscation processor renamed or removed a class, we couldn’t find the telltale signs of such an activity - no exceptions at all.

Desired Traits for Modern Software

Wed, 19 Jan 2011 00:00:00 +0000

In my last entry, I mentioned that we need to look at more than correct execution to determine if the software our dev teams produce is actually high quality software and this requires us to determine what traits we want the code to have. When asked, most people will probably include something about scalability, whereby they want the most efficient memory footprint or to maximize the concurrent number of jobs in process at any given time. This is a noble goal to have but let’s be real - most of the software we write won’t suffer from the scale issues of eBay, Google or Facebook. Even in that list, scalability as we’ve defined it wasn’t an issue at first so it’s probably not a good quality to include up front.

Software QA: What the heck are they verifying?

Sun, 09 Jan 2011 00:00:00 +0000

Ask layperson what a quality assurance tester does and, as you can guess, the answer won’t clarify the situation much - they test the software to make sure it has some level of quality. That answer didn’t really clear up the situation any and it all revolves around the fact that we haven’t defined what quality is. It’s a very interesting question to pose and one that I’m not sure has a straight forward answer.

What's a Caffeinated Idea?

Tue, 04 Jan 2011 00:00:00 +0000

Truth be told, the name was a domain that I registered sometime in 2005 when I was trying to build a tech startup in Milwaukee, WI at night while working a boring IT job during the day to support myself. I was naive back then and thought that there was no way I could fail in this venture because, let’s face it, I was the best developer in Milwaukee. I spent hours at night tinkering with python, MySQL and JavaScript in an insane attempt to create the next Google.