Feed

Six and a half ridiculous things to do with Quarkus

Wed, 17 Jun 2026 00:00:00 GMT

Let’s abuse the tools! Everyone knows Quarkus is computationally efficient, expressive, and rock-solid for production. But did you know that we can use Quarkus efficiency to build applications that shouldn’t go anywhere near production? In this demo-driven session, Holly will put the joy into “developer joy”. She’ll show you all sorts of things you can do with Quarkus that you probably shouldn’t:

Build an LLM-powered app that’s guaranteed to hallucinate, because you can do more than you think with guardrails, and truth is so boring
Write your business code in rockstarlang, because everything should be a hair metal ballad
Use Minecraft as your observability client, because the LGTM stack doesn’t have enough explosions
Write a CLI for generating memes faster, because everything is better on the command-line
Benchmark an application against a grapefruit, because metric units aren’t tasty

Business value? Learning? If you insist. As well as absurd demos, you’ll leave this session with a deeper understanding of how to get the most out of Quarkus and Java. There will be new Java 25 language features, Quarkus best practices, powerful integrations, and nifty use cases alongside the silly explosions and grapefruit.

When benchmarks go bad - what I learned from measuring performance wrong

Thu, 04 Jun 2026 00:00:00 GMT

The world of performance analysis is littered with flawed claims, cognitive biases, dangerous intuitions, and beguiling fallacies. Sadly, Holly has been guilty of all of the above! Repeatedly. But this is a no-judgement zone. Some measurement anti-patterns are subtle, and some are downright counter-intuitive. In this talk, Holly will explain why measuring performance is important, and talk through some of the ways it can go wrong. That would be depressing if that was all there was, so she’ll also introduce a toolbox of questions and principles that you can use to improve the performance of your own applications.

These include:

How to set up a test system
Recommended load generators
The USE method

Things You Thought You Didn’t Need To Care About That Have a Big Impact On Your Job

Wed, 13 May 2026 00:00:00 GMT

Development is about working with computers, right? Well, not quite. Development is all about working with computers ( easy), and working with people (hard). Oh, and it’s about physics. Things like the speed of light and thermodynamics influence APIs, because they influence hardware and networking. If, like Holly, you slept through statistics modules in university, it will be a surprise to discover how statistics has changed our development workflows. Finally, we mustn’t forget economics. The end of zero-interest-rates has changed the employment landscape for many of us. In this wide-ranging talk, Holly will cover why the end of Moore’s law means we might finally need to get to grips with concurrent programming, why is Loom a good idea now when green threads were a bad idea, why is AOT a good idea now when it used to be a bad idea, and how much you should care about business studies, finance, and statistics.

Five (and a half) Cool Things You Can Do With Quarkus

Wed, 06 May 2026 00:00:00 GMT

Save the world.
Lightning-quick CLIs.
Embed WASM.
Run Spring code.
AI, for real.

5½. Use Minecraft as an observability client. Ok, this one isn’t cool, it’s stupid, but Holly’s done it anyway.

In this demo-driven session, Holly will show some of her favorite Quarkus capabilities. Some are established and important, some are less well-known, and some are plain silly. But they’re all cool. Whether you’re new to Quarkus or an experienced user, you’ll discover something you didn’t know.

When benchmarks go bad - what I learned from measuring performance wrong

Wed, 06 May 2026 00:00:00 GMT

These include:

How to set up a test system
Recommended load generators
The USE method

Kodsnack 701 - The chicken would explode

Tue, 05 May 2026 00:00:00 GMT

Fredrik chats to Holly Cummins about using Minecraft for observability, other amazing Quarkus tricks, and the value of joy at work.

Recorded during Øredev 2025.

Thank you Cloudnet for sponsoring our VPS!

Comments, questions or tips? We a re @kodsnack, @tobiashieta, @oferlund and @bjoreman on Twitter, have a page on Facebook and can be emailed at info@kodsnack.se if you want to write longer. We read everything we receive. If you enjoy Kodsnack we would love a review in iTunes!

We're Gonna Build a Benchmark – Lyrics

Thu, 30 Apr 2026 00:00:00 GMT

I was inspired by Dylan Beattie’s “We’re Gonna Build a Framework” (go watch it, it’s so good!) to make a song called “We’re Gonna Build a Benchmark”. (Dylan was of course, himself, riffing on Billy Joel’s “We Didn’t Start the Fire,” but I think I’ve listened to Dylan’s version more than Billy Joel’s, so Dylan’s are the lyrics I hear in my head.) Sadly, I lack Dylan’s musical talent, so I had to use Suno for the music production. It’s not all cheating, because I wrote the lyrics myself.

The rendered song is here, on Suno.

Lyrics

Throughput, build time, RSS should be fine Latency, P99 Warmups, ready line RAPL, energy CPU, wrong frequency

Cold start, Hot cache, Page fault, disk thrash Sync cost, lock-free For latency Soft swap, core crash Queue depth, load smash Profile, flame graph

We’re gonna build a benchmark ‘cos we wanna use one, but don’t wanna choose one, We’re gonna build a benchmark ‘cos we wanna use one, but don’t wanna choose one,

Heap size, lazy threads Pool size bikesheds Metrics, tracing, Otel excess Pinned cores Sync chores Twist the knob, NUMA on the floor

I/O bound, rebound scores on the ground numbers are not sound Cache pop, head drop, Works on my machine, is it really clean?

Change one little flag Watch the numbers sag Tune, run, tweak, repeat Share the test and shriek Read rows, write rows Stale container woes CPU GPU, what else do I have to say?

Wrk2, hyperfoil, Cache hit, slashdot Mutex, deadlock A/B, baby, Share the code, Why’s it slow in this mode? Data, state-a, I can’t take it anymore

We’re gonna build a benchmark ‘cos we wanna use one, but don’t wanna choose one, We’re gonna build a benchmark ‘cos we wanna use one, but don’t wanna choose one.

Things You Thought You Didn’t Need To Care About That Have a Big Impact On Your Job

Fri, 24 Apr 2026 00:00:00 GMT

How We Benchmarked Quarkus: Patterns and anti-patterns

Mon, 20 Apr 2026 00:00:00 GMT

Holly, Eric, and Franz discuss some of the lessons learned in creating the new Quarkus benchmark suite.

Java and Web – airhacks.fm podcast with Adam Bien

Mon, 13 Apr 2026 00:00:00 GMT

Six and a half ridiculous things to do with Quarkus

Wed, 01 Apr 2026 00:00:00 GMT

Build an LLM-powered app that’s guaranteed to hallucinate, because you can do more than you think with guardrails, and truth is so boring
Write your business code in rockstarlang, because everything should be a hair metal ballad
Use Minecraft as your observability client, because the LGTM stack doesn’t have enough explosions
Write a CLI for generating memes faster, because everything is better on the command-line
Benchmark an application against a grapefruit, because metric units aren’t tasty

Things You Thought You Didn’t Need To Care About That Have a Big Impact On Your Job

Tue, 31 Mar 2026 00:00:00 GMT

Fixing The Open Source Bus Number

Thu, 19 Mar 2026 00:00:00 GMT

Open source is a marvellous multiplier which amplifies our collective productivity by allowing us to code on the shoulders of giants. It has clear benefits for business continuity.

But things can go wrong. Many critical projects have just one or two maintainers (we’ve all seen the xkcd cartoon about modern digital infrastructure, haven’t we?). Sometimes those maintainers no longer have time, or burn out. At the other end of the spectrum, even projects with many active contributors can grind to a halt with the contributors all work for the same employer, and that employer changes their business priorities. Investment can dry up or the licence can change. After all, the business environment for open source is tough these days.

What can be done?

The classic solution for increasing a team’s bus number is to use pair programming to share knowledge. In the asynchronous and distributed world of open source, pair programming can work, but it’s harder – and it can’t fix changing business realities or hostile licence changes.

This talk will discuss some of the strategies used by the Quarkus team to help ensure our open source community is prepared … for almost anything. These include

Using GitHub organisations to allow independent contributions with a ‘safety’ backup if maintainers disappear
Productising dependencies so that anything in the stack can be rebuilt to address vulnerabilities; this helps with security, and also helps with investment
Joining the Commonhaus foundation to ensure Quarkus can’t ever be owned by a single company

Some of these have been easy, some have been tough (the logistics of moving to a foundation are not for the faint-hearted). In this talk, Holly and Sanne will reflect on what’s worked well, what’s transferrable, and how all of us can be prepared for the future of open source.

The free-lunch guide to idea circularity

Wed, 18 Mar 2026 00:00:00 GMT

Why do the same ideas keep coming round again, and again? Why is it that sometimes an idea sticks, and sometimes it just fades away? Why is it that no technology, or pattern, or method, is ever quite as good as we’re told it’s going to be? Why are we working harder, even though AI is supposed to be doing the work for us? Is debt inevitable? How do external factors influence the day-to-day reality of our jobs? In this wide-ranging session, Holly will discuss interest rates, ahead-of-time compilation, how venture capital works, why our codebases might be feeling the AI hangover, and the future of developer jobs.

Developer Journey: Accelerating Developer Productivity

Tue, 17 Mar 2026 00:00:00 GMT

In 2026, we’ve learned not to measure developer productivity by lines of code. What actually matters is the velocity of the “inner loop” and getting to production. Agentic AI and hyper-distributed deployment topologies solve many problems, but developers still face many of the same frictions – cognitive overload, code overload, and configuration overload. This session mixes a high level strategic overview of the modern developer journey with live demos of how IBM’s modern frameworks bring the developer joy

It will cover:

Why developer productivity is about more than outputs
Standard First Architecture: Why relying on Jakarta EE and MicroProfile creates portable, future-proof systems.
Observability by Design: Moving beyond simple logging to a standardized telemetry approach that provides instant feedback.
How to identify and eliminate “toil” in your specific development lifecycle.

Quarkus has great performance – and we have new evidence

Mon, 02 Mar 2026 00:00:00 GMT

Quarkus: Supersonic Subatomic Java

From Quantum Physics to Quarkus – airhacks.fm podcast with Adam Bien

Thu, 26 Feb 2026 00:00:00 GMT

Security & DevEx: Can We Have Both? • Abby Bangser, Adrian Mouat & Holly Cummins • GOTO 2025

Wed, 11 Feb 2026 00:00:00 GMT

This interview was recorded at GOTO Copenhagen 2025. #GOTOcon #GOTOcphhttps://gotocph.comAbby Bangser - Platform Engineering Insights from Syntasso deliverin…

Quarkus in Action • GOTO Book Club

Tue, 03 Feb 2026 00:00:00 GMT

This interview was recorded for the GOTO Book Club.http://gotopia.tech/bookclub

Check out more here:https://gotopia.tech/episodes/416Martin

Six and a half ridiculous things to do with Quarkus

Tue, 27 Jan 2026 00:00:00 GMT

Build an LLM-powered app that’s guaranteed to hallucinate, because you can do more than you think with guardrails, and truth is so boring
Write your business code in rockstarlang, because everything should be a hair metal ballad
Use Minecraft as your observability client, because the LGTM stack doesn’t have enough explosions
Write a CLI for generating memes faster, because everything is better on the command-line
Benchmark an application against a grapefruit, because metric units aren’t tasty

Foojay Podcast #89: Quarkus and Agentic Commerce

Mon, 26 Jan 2026 00:00:00 GMT

Things Software Developers Think They Don’t Need to Care about, But Can Impact Their Job

Thu, 01 Jan 2026 00:00:00 GMT

Holly Cummins gave a keynote at Goto Copenhagen where she urged developers to care about overlooked issues that shape their work. She warned of unintended consequences of design decisions, promoted sy

Five (and a half) Cool Things You Can Do With Quarkus

Wed, 05 Nov 2025 00:00:00 GMT

Save the world.
Lightning-quick CLIs.
Embed WASM.
Run Spring code.
AI, for real.

5½. Use Minecraft as an observability client. Ok, this one isn’t cool, it’s stupid, but Holly’s done it anyway.

Developer Joy - The New Paradigm

Wed, 22 Oct 2025 00:00:00 GMT

Ever been told “work is not a place to be happy?” It’s not true! As developers, our job is to be productive, and we’re most productive when we’re in flow, doing friction-free creation. How do we embrace the happiness paradigm, without throwing out the old ones we also need, such as efficiency and safety? In this talk, Holly will explain how our industry is changing, and present a number of productivity tips for the lazy, happy, developer.

Using LangChain4j to Make Your Java Apps Smarter

Tue, 14 Oct 2025 00:00:00 GMT

AI isn’t just for coding assistants and python programmers. The real power of LLMs comes from integrating them into applications, connected to real data and real business logic. LangChain4j is emerging as a de facto standard for LLM integration in Java. In this demo-driven talk, Holly will explore a range of cool LangChain4j capabilities, such as type safe object mapping, stateful context, agents, RAG, guard rails, and fault tolerance. (The demos will use Quarkus, but LangChain4j can be used with any framework.)

Things You Thought You Didn’t Need To Care About That Have a Big Impact On Your Job

Thu, 09 Oct 2025 00:00:00 GMT

Six and a half ridiculous things to do with Quarkus

Thu, 09 Oct 2025 00:00:00 GMT

Build an LLM-powered app that’s guaranteed to hallucinate, because you can do more than you think with guardrails, and truth is so boring
Write your business code in rockstarlang, because everything should be a hair metal ballad
Use Minecraft as your observability client, because the LGTM stack doesn’t have enough explosions
Write a CLI for generating memes faster, because everything is better on the command-line
Benchmark an application against a grapefruit, because metric units aren’t tasty

Things You Thought You Didn’t Need To Care About That Have a Big Impact On Your Job

Thu, 02 Oct 2025 00:00:00 GMT

Things You Thought You Didn’t Need To Care About That Have a Big Impact On Your Job

Thu, 18 Sep 2025 00:00:00 GMT

Optimizing Cloud Native Java • GOTO - The Brightest Minds in Tech

Fri, 08 Aug 2025 00:00:00 GMT

This interview was recorded for the GOTO Book Club.http://gotopia.tech/bookclub

Read the full transcription of the interview here.

LangChain4j, Java, and You

Fri, 11 Jul 2025 00:00:00 GMT

If you’re not experimenting with AI, were you even developing in 2025? LangChain4j is emerging as a de facto standard for LLM integration in Java. In this demo-driven talk, Holly will explore a range of cool LangChain4j capabilities, such as type safe object mapping, stateful context, agents, RAG, guard rails, and fault tolerance.

The ELUSIVE ART of User-Focused Simplicity In Software

Fri, 27 Jun 2025 00:00:00 GMT

This clip from The Engineering Room features Dave Farley and Holly Cummins, Senior Principal Software Engineer at Red Hat, as they explore The ELUSIVE ART of…

Aflevering 99: The Hidden Cost of Doing Nothing: Hunting Cloud Zombies | De Nederlandse Kubernetes Podcast

Tue, 24 Jun 2025 00:00:00 GMT

Holly explains that waste in our infrastructure is often easier to solve than we think. With simple automation, time-bound resources, or tools like Dailyclean, it’s possible to make an impact without large investments.

Productivity Through Play: Why Messing Around Makes Better Software Engineers

Fri, 20 Jun 2025 00:00:00 GMT

In this podcast, Shane Hastie, Lead Editor for Culture & Methods, spoke to Holly Cummins about productivity in creative knowledge work like software engineering.

Developer Joy: A Better Way to Boost Developer Productivity

Wed, 04 Jun 2025 00:00:00 GMT

Joy isn’t a distraction from productivity. Learn how to reclaim developer satisfaction and boost output by embracing curiosity, minimizing friction, and giving ourselves a break.

Efficiency and Rock 'n’ Roll (Really!)

Tue, 27 May 2025 00:00:00 GMT

Is efficiency rock and roll? You might think the answer is “no, definitely not, no way.” Efficiency is about doing more with less, and rock n’ roll is all about turning it up to eleven and doing more with more. Efficiency is about being responsible, and rock n’ roll is about being irresponsible. But wait! High overheads are not rock n roll, and they’re not efficient. Spending time on boring tasks is neither rock n roll, nor efficient. We’re spending way too much time writing and running code that doesn’t spark joy. In this talk, Holly will discuss practical patterns for wringing maximum value out of our code, for minimum effort. One of the techniques is sleeping! It turns out, both people and computers should be lazier, and that’s definitely rock n roll.

Java meets AI: Build LLM-Powered Apps with LangChain4j

Thu, 08 May 2025 00:00:00 GMT

Join us for a guided tour through the possibilities of the LangChain4j framework! Chat with virtually any LLM provider ( OpenAI, Gemini, HuggingFace, Azure, AWS, …)? Generate AI images straight from your Java application with Dall-E and Gemini? Have LLMs return POJOs? Interact with local models on your machine? LangChain4j makes it a piece of cake! We will explain the fundamental building blocks of LLM-powered applications, show you how to chain them together into AI Services, and how to interact with your knowledge base using advanced RAG.

Then, we take a deeper dive into the Quarkus LangChain4j integration. We’ll show how little code is needed when using Quarkus, how live reload makes experimenting with prompts a breeze and finally we’ll look at its native image generation capabilities, aiming to get your AI-powered app deployment-ready in no time. By the end of this session, you will have all the technical knowledge to get your hands dirty, along with plenty of inspiration for designing the apps of the future.

Things You Thought You Didn’t Need To Care About That Have a Big Impact On Your Job

Wed, 07 May 2025 00:00:00 GMT

Create AI-Infused Apps with LangChain4j (and Quarkus) (Mini Lab)

Wed, 07 May 2025 00:00:00 GMT

Generative AI has taken the world by storm, and it seems like every executive leader out there is telling us “regular” Java devs to “add AI” to our apps. Does that mean we need to drop everything we’ve built and become data scientists instead now?

Fortunately, we can infuse AI models built by actual AI experts into our applications in a fairly straightforward way. We promise it’s not as complicated as you might think! Thanks to the ease of use and superb developer experience of Quarkus and the nice AI integration capabilities that the LangChain4j libraries offer, it becomes trivial to start working with AI and make your stakeholders happy.

In this session, you’ll explore a variety of AI capabilities. We’ll start from the Quarkus DevUI where you can try out AI models before writing any code. Then we’ll get get into the code and explore LangChain4j features such as prompting, chaining, and preserving state; agents and function-calling; enriching your AI model’s knowledge with your own documents using retrieval augmented generation (RAG); and discovering ways to run (and train) models locally using tools like Ollama and/or Podman AI Lab. In addition, we’ll take a look at observability and fault tolerance of the AI integration. We might even try some new features, such as MCP.

Come to this session to learn how to build AI-infused applications in Java. This is also an opportunity to provide feedback to the maintainers of these projects and contribute back to the community.

How Developers Can Eliminate Software Waste and Reduce Climate Impact

Mon, 28 Apr 2025 00:00:00 GMT

High performance and sustainability correlate; making software go faster by improving the efficiency of algorithms can reduce energy requirements, Holly Cummins said at QCon London. She suggested swit

The Evolution Of Kubernetes Workload Patterns

Wed, 23 Apr 2025 00:00:00 GMT

Kubernetes workloads differ because they represent applications and services deployed within a cluster encompassing one or more components.

How AI will change software engineering

Mon, 14 Apr 2025 00:00:00 GMT

No one can agree how AI will change software engineering. Here’s how to prepare anyway.

The Efficiency Paradox and How to Save Yourself and the World

Mon, 07 Apr 2025 00:00:00 GMT

Inefficiency is ruining our planet and our lives. Efficiency is ruining our happiness, and weirdly, it’s also ruining our efficiency. Heeeeelppp!? What’s a techie to do? In this talk, Holly walks through a range of techniques that can be used to find and eliminate software waste and reduce climate impacts. For example, zombie servers and slow code are a big climate problem, but the vrrrooom model gives us double-win hope.

But machine efficiency isn’t much use without human efficiency. The good news is that there isn’t as much of a tradeoff as you might think between these two types of efficiency. Sorting out machine efficiency often helps humans, too. For example, the Quarkus Java framework uses many interesting waste-reduction techniques. These optimisations have the dual benefit of speeding up computers, and also speeding up people.

So far, so good, but we need to be careful we don’t end up accidentally optimising the wrong things, and making stuff worse. 100% utilisation is not sustainable for either humans, or people. It’s not even very efficient (remember the paradox part?). It turns out, thinking about efficiency needs systems thinking. Holly will draw lessons from performance engineering, mathematics,economics, and psychology to give guidance on how to achieve more by doing less.

Productivity is Messing Around and Having Fun

Mon, 07 Apr 2025 00:00:00 GMT

Developer satisfaction, developer joy, and business results are strongly correlated. Developer productivity frameworks like SPACE recognize this, with Satisfaction being a key metric. And yet - our jobs are frustrating, filled with mindless work, and free of joy. How do we fix that? Is annoying waste inevitable? Can developer performance be tuned? Are productivity measurements helping or hurting us? And how can you persuade management to invest in boredom?

Except… it’s not the boss we need to convince. Sometimes, it’s ourselves. We’re so in the habit of running to meet deadlines, running from one problem to the next, of aiming for 100% efficiency, we’re the first ones to say “I don’t have time for fun”. In this talk, Holly and Trisha will try to convince you that fun isn’t a luxury, but a necessity.

Holly is an expert on play at work, unwise automations, and polar bears. Trisha is an expert on performance tuning, tooling and productivity. Come to this talk to find out what these topics have in common.

Practical Zombie Hunting for Kubernetes Users

Thu, 03 Apr 2025 00:00:00 GMT

Zombies? Yup, zombies. Zombies are servers which aren’t doing useful work. They’re everywhere, costing money, eating electricity, and belching carbon. And they’re useless! Sadly, the cloud has not helped our zombie problem, and even Kubernetes hasn’t helped.

One of the reasons zombies don’t get switched off is that no one knows they’re there. So how do we get rid of our pesky zombies? In this talk, Holly will explain the underlying technical and organisational factors that lead to zombies, and introduce a range of real-world zombie-hunting strategies. These include getting to grips with elasticity and utilisation, LightSwitchOps, FinOps, and the eco-monkey (it’s like the chaos monkey, but greener). Technologies covered include absurdly simple scripts, DailyClean, Kruize Autotune, and Backstage.

The Weight of Data: Rethinking Cloud-Native Systems for the Age of AI

Thu, 03 Apr 2025 00:00:00 GMT

The explosion of data and the rise of AI agents have brought cloud-native systems to a critical tipping point. While the CNCF has driven significant advancements, the challenges of fault tolerance and state management in the age of AI remain unresolved both at the core and the edge.

This keynote examines how data gravity, distributed environments, and the complexity of AI workloads demand a fundamental shift from stateless architectures to intelligent, state-aware platforms. This provides a unique opportunity for the CNCF and its ecosystem to pave the way for a new generation of cloud-native, resilient, cost efficient, scalable, and AI-ready systems, while tackling the pressing challenges that lie ahead.

Kubernetes drives AI scale with cloud-native power - SiliconANGLE

Thu, 03 Apr 2025 00:00:00 GMT

Kubernetes plays a key role in scaling AI, bridging stateless design with stateful needs while supporting cloud-native, edge and sustainable strategies.

theCUBE at KubeCon+CloudNativeCon EU 2025

Thu, 03 Apr 2025 00:00:00 GMT

Holly Cummins, Senior Principal Software Engineer, Red Hat, &Vincent Caldeira, CTO APAC, Red Hat, join theCUBE for this segment of KubeCon+CloudNativeCon EU …

Environment Variables: Sustainable AI Progress

Thu, 20 Mar 2025 00:00:00 GMT

For this 100th episode of Environment Variables from the Green Software Foundation, guest host Anne Currie is joined by Holly Cummins, senior principal engineer at Red Hat, to discuss the intersection of AI, efficiency, and sustainable software practices. They explore the concept of “Lightswitch Ops”—designing systems that can easily be turned off and on to reduce waste—and the importance of eliminating zombie servers. They cover AI’s growing energy demands, the role of optimization in software sustainability, and Microsoft’s new shift in cloud investments. They also touch on AI regulation and the evolving strategies for balancing performance, cost, and environmental impact in tech.

Optimizing Cloud Native Java • GOTO Book Club

Thu, 13 Mar 2025 00:00:00 GMT

This interview was recorded for the GOTO Book Club.

#GOTOcon #GOTObookclub

http://gotopia.tech/bookclub

Read the full transcription of the interview here: https://gotopia.tech/episodes/363

Zero Waste, Radical Magic, and Italian Graft – Quarkus Efficiency Secrets

Tue, 28 Jan 2025 00:00:00 GMT

What makes a platform efficient? Is it how quickly code executes, or is it how quickly developers can use it to solve problems? Quarkus makes both people and hardware more efficient. That’s cool, but how does it work? Usually, we expect to trade-off developer experience against runtime efficiency. In this session, Holly will dive into some of the technical underpinnings of Quarkus’s efficiency. She’ll give advice for those using or considering Quarkus - should you be doing reactive programming? Do native binaries run fastest? She’ll also share general principles and transferrable discoveries from the engineering team, such as knowing what you’re optimising for, avoiding cache pollution, and how to be static-but-dynamic.

The Efficiency Paradox and How to Save Yourself and the World

Fri, 13 Dec 2024 00:00:00 GMT

Inefficiency is ruining our planet and our lives. Efficiency is ruining our happiness, and weirdly, it’s also ruining our efficiency. Heeeeelppp!? What’s a techie to do? Zombie servers and slow code are a big climate problem, but the vrrrooom model gives us double-win hope. Holly walks through a range of techniques that can be used to find and eliminate software waste and reduce climate impacts.

But machine efficiency isn’t much use without human efficiency. Surprisingly, sorting out machine efficiency often helps humans, too. For example, the Quarkus Java framework uses many interesting waste-reduction techniques. These optimisations have the dual benefit of speeding up computers, and also speeding up people.

So far, so good, but we need to be careful we don’t end up accidentally optimising the wrong things, and making stuff worse. 100% utilisation is not sustainable for either humans, or people. It’s not even very efficient (what!?). Holly gives a brief tour of the brain’s default mode network, the latest business research, and queueing theory, to show why we actually achieve more when we do a bit less.

Tradeoffs, Bad Science, and Polar Bears – The World of Java Optimisation

Fri, 13 Dec 2024 00:00:00 GMT

Welcome to the Java optimisation jungle. Why can’t we “just make it go faster”? It turns out, in most cases, we need to first work out “faster for whom?” and “why do we want to go faster?” and “what even is faster?” This talk introduces the basic principles of optimisation, before bouncing through the pitfalls of optimisation; why the exact same techniques which make Quarkus rocket-fast used to be a terrible idea fifteen years ago, why fast benchmarks make for slow programs, and why even though it can be easy to get wrong, optimisation really really matters. Along the way we’ll talk about measuring things, bad advice, garbage collection, and climate change.

The Efficiency Paradox and How to Save Yourself and the World

Tue, 10 Dec 2024 00:00:00 GMT

So far, so good, but we need to be careful we don’t end up accidentally optimising the wrong things, and making stuff worse. 100% utilisation is not sustainable for either humans, or people. It’s not even very efficient (what!?). Holly gives a brief tour of the brain’s default mode network, the latest business research, and queueing theory, to show why we actually achieve more when we do a bit less.

The Efficiency Paradox and How to Save Yourself and the World

Fri, 06 Dec 2024 00:00:00 GMT

So far, so good, but we need to be careful we don’t end up accidentally optimising the wrong things, and making stuff worse. 100% utilisation is not sustainable for either humans, or people. It’s not even very efficient (what!?). Holly gives a brief tour of the brain’s default mode network, the latest business research, and queueing theory, to show why we actually achieve more when we do a bit less.

Tradeoffs, Bad Science, and Polar Bears – The World of Java Optimisation

Thu, 05 Dec 2024 00:00:00 GMT

The Engineering Room with Dave Farley

Sun, 01 Dec 2024 00:00:00 GMT

Listen to Dave Farley’s The Engineering Room with Dave Farley podcast on Apple Podcasts.

Leverage LLMs in Java with LangChain4j and Quarkus

Fri, 22 Nov 2024 00:00:00 GMT

In this session, we’ll explore how to infuse AI capabilites into Java applications, using LangChain4j and its Quarkus integration. We’ll start from the Quarkus DevUI where you can try out AI models even before writing any code. Then we’ll explore LangChain4j features such as prompting, chaining, and preserving state; agents and function-calling; enriching your AI model’s knowledge with your own documents using retrieval augmented generation (RAG); and discovering ways to run (and train) models locally using tools like Ollama and/or Podman AI Lab. In addition, we’ll take a look at observability and fault tolerance of the AI integration and compile the app to a native binary.

Fortunately, we can infuse AI models built by actual AI experts into our applications in a fairly straightforward way, thanks to some new projects out there. We promise it’s not as complicated as you might think! Thanks to the ease of use and superb developer experience of Quarkus and the nice AI integration capabilities that the LangChain4j libraries offer, it becomes trivial to start working with AI and make your stakeholders happy :)

In this session, you’ll explore a variety of AI capabilities. We’ll start from the Quarkus DevUI where you can try out AI models even before writing any code. Then we’ll get our hands dirty with some code and exploring LangChain4j features such as prompting, chaining, and preserving state; agents and function-calling; enriching your AI model’s knowledge with your own documents using retrieval augmented generation (RAG); and discovering ways to run (and train) models locally using tools like Ollama and/or Podman AI Lab.

Less waste, more joy, and a lot more green – How Quarkus makes Java better

Wed, 06 Nov 2024 00:00:00 GMT

Quarkus makes both people and hardware more efficient. That’s cool, but how does it work? Usually, we expect to trade-off developer experience against runtime efficiency. In this session, Holly will dive into some of the technical underpinnings of Quarkus’s efficiency. She’ll give advice for those using or considering Quarkus - should you be doing reactive programming? Do native binaries run fastest? In this under-the-hood session, she’ll also share general principles and transferrable discoveries from the engineering team, such as knowing what you’re optimising for, avoiding cache pollution, and how to be static-but-dynamic.

The Power of LLMs in Java – Leveraging Quarkus and LangChain4j

Wed, 23 Oct 2024 00:00:00 GMT

Generative AI has taken the world by storm over the last year, and it seems like every executive leader out there is telling us “regular” Java application developers to “add AI” to our applications. Does that mean we need to drop everything we’ve built and become data scientists instead now? Fortunately, we can actually infuse AI models built by actual AI experts into our applications fairly straightforwardly, thanks to some new projects out there. We promise it’s not as complicated as you might think! Thanks to the ease of use and superb developer experience of Quarkus and the nice AI integration capabilities that the LangChain4j libraries offer, it becomes trivial to start working with AI and make your stakeholders happy :) In this session, you’ll explore a variety of AI capabilities. We’ll start from the Quarkus DevUI where you can try out AI models even before writing any code. Then we’ll get our hands dirty with some code and exploring LangChain4j features such as prompting, chaining, and preserving state; agents and function-calling; enriching your AI model’s knowledge with your own documents using retrieval augmented generation (RAG); and discovering ways to run (and train) models locally using tools like Ollama and/or Podman AI Lab. In addition, we’ll take a look at observability and fault tolerance of the AI integration and compile the app to a native binary.

Refactoring to Kube-native Java with Quarkus

Wed, 23 Oct 2024 00:00:00 GMT

Quarkus was created to enable Java developers to deliver applications for a modern, cloud-native world. A Kubernetes-native Java framework, Quarkus is tailored for the Java VM and GraalVM native image, and crafted from best-of-breed Java libraries and standards. Join us as we explore the technical capabilities, critical use cases, and benefits that make Quarkus ideal for Kubernetes and serverless environments while offering developers a framework to address a wider range of distributed application architectures. We’ll even throw in a demonstration of Quarkus in action.

Choosing the right Java runtime for the job

Tue, 22 Oct 2024 00:00:00 GMT

Successful digital transformation requires business and IT systems to be secure, sustainable, and minimize technical debt. Technical debt stifles innovation and burdens application teams with security weaknesses and inefficient resource utilization. Select the right Java runtime for your application needs to begin addressing technical debt and delivering new cloud native applications with the frameworks and IDEs of choice.

The future of development – Are our jobs getting harder or easier?

Sat, 19 Oct 2024 00:00:00 GMT

In the early days of computing, software developers had to encode their programs on punch cards, and carry the bits around by hand. By the 1970s they had the joy of keyboards and displays. Sadly, those oppressed developers had to wear flare trousers, work in C, and manage their own memory. By 2000 the memory thing was sorted out, but the trousers were arguably worse, and developers had to maintain their own hardware. Now, our hardware is someone else’s problem, everything is open source, AI can write our code, and we have a choice whether to work in Java, Kotlin, Javascript, Ruby, Go, Python, Rust, or Typescript. On the other hand, now we have yaml. And many of us find we have to work in ALL of Java, Kotlin, Javascript, Ruby, Go, Python, Rust, or Typescript, all while managing a cloud, becoming security experts, learning CSS, and debugging the code our AI “helper” wrote. And we’re supposed to be full-stack, and shifting left, and building up open source in our free time. Are we living the dream or sinking under cognitive overload?

Zombie Menace, Vampire Peril

Wed, 16 Oct 2024 00:00:00 GMT

It’s the spookiest time of year! You might think spookiness and tech are unrelated, but you’d be wrong. Well, actually, you’d probably be right, but I’m not going to let that stop me writing Hallowe’en-themed talks about getting cloud native wrong, and Hallowe’en-themed talks about the importance of TestContainers cleanup logic.

This year, I’ve been thinking about zombies and vampires. Zombies in movies are scary, but (spoiler) they don’t exist. Same for vampires.

But in the tech world, these monsters do exist. And they’re pretty bad. Maybe not ripping-your-head off bad, but definitely bad enough to be worth exorcising.

Vampire code

Let’s start off by talking about code. As developers, we work really really hard to create code. So many of us assume code must be an asset. It’s not.

Code is a liability. Every bit of code we produce needs to be maintained. Code increases the attack surface of applications, creates cognitive load for developers, and slows down compilation. Ugh. Who’d want code?

Martijn Verburg once said that he paid his junior developers by how much code they wrote, and his senior developers by how much code they deleted. People sometimes compare working with an LLM to pair programming with a very enthusiastic junior developer. The model will get excited by your request and produce as much code as it possibly can. I suspect something in the mathematics of how these models are trained biases them towards verbosity. I saw this anecdotally when I generated a Quarkus application and 70% of the generated code was unnecessary. The Hibernate with Panache libraries eliminate the need for boilerplate code, but the AI assistant put all the boilerplate in anyway.

Statistical analysis of codebases seems to confirm my intuition. A study by GitClear found that codebases today have more duplicated code than they did before developers started using tools like CoPilot and ChatGPT. They also found a reduction in the number of commits which deleted code. As developers, we spend far more time reading code than writing code, so this excess code becomes an ongoing drain on developer time.

Code can be vampire code even when it actually is needed in the codebase. All but the smallest libraries usually include files and functions which a particular consumer application might not use. Even though they’re unused, these dormant bits of code still have a cost. They increase the size of the packaged application, which means increased network traffic for users. This is extra bad for Javascript libraries, which will be downloaded every single time a web page loads. The code bloat doesn’t just make page load slower, it uses energy.

Extra code can also mean extra memory footprint; for example, the Java runtime will load unused database support classes into memory as part of the Hibernate bootstrap process. The classes will then be unloaded, but by then it’s too late; the peak memory requirement has been set high.

Extra code can even slow applications down. For example, in Java, the process of invoking an interface method on an object is slower if multiple possible implementations exist on the classpath. If the JVM has to choose between several implementations, that’s called megamorphic dispatching. It’s slower than the single implementation case, monomorphic dispatching.

The good news is that there is a solution. For Javascript, where extra code has such an obvious bad impact, most modern build tooling will do a process called tree-shaking, which strips unused code. Historically, Java did not have a similar process, but that’s now changing. For example, building a GraalVM native binary will aggressively eliminate unused code. Project Leyden may also some tree-shaking ideas to the JVM itself.

At a level above the runtime, the Quarkus architecture is designed to reduce vampire code. Quarkus applies a number of build-time optimisations to applications, and libraries can participate in this optimisation process through extensions.
For example, all of those undead database support classes in Hibernate? The Quarkus Hibernate extension makes sure they never make it into the built application. Think of Quarkus’s build architecture as an extensible wooden stake.

Zombie servers

Vampire code happens when a portion of an application’s codebase is unneeded. Zombies happen when the entire application is unneeded. By extension, not only is the application wasting resources, so is all the infrastructure running it.

How could a whole application be unused? Usually, the root cause is forgetfulness: individual forgetfulness, and also organisational forgetfulness. It may be that something was originally well-used, but then processes changed. Or perhaps a prototype was put into staging, and never progressed to production, but never got torn down, either.

Over-provisioning also causes zombies. Organisational incentive structures encourage over-provisioning, because no one wants to be the person who provisioned too little infrastructure and caused an outage. To avoid this career-limiting situation, many people will err on the side of caution and configure far too much capacity.

How bad is the zombie menace? It’s bad.

NRDC research concluded that the average server was running at 12-18% of maximum capacity. That wouldn’t be terrible, except a machine running with such low utilization still uses 30-60% of its maximum power. (This ratio is called “energy proportionality,” if you want to bring out the technical terms.) Things are even more disproportionate when we look at the hardware side. Building an unused server emits exactly the same amount of carbon and consumes the same amount of natural resources as building a well-utilised server. The cost to us and the planet is the same, but the value is far far lower.

A 2017 survey from the Anthesis Institute found that, over a six month period, 29% of servers had utilisation less than 5%. A quarter of the servers had no utilization at all; they were completely unused.

That quarter figure is the average over a large sample, and included a number of different companies. In some unlucky organisations, the proportion of zombies can be higher. Like, much much higher. Building Green Software shares the story of a VMWare data centre in Singapore. The data centre was being physically moved, and so before doing the move, the team checked what was running on the machines. Rather incredibly, 66% of all the host machines were zombies. That’s a full-on Twenty Eight Days Later scenario, in which the zombies outnumber the value-creating machines.

Hunting down zombie servers isn’t trivial; kind of by definition, forgotten servers are hard to find. The reason VMWare discovered them was because the physical move forced a virtual clear-out. Otherwise, who knows how long the useless servers might have hung around? Regular audits of an estate can help, but, regrettably, they are pretty boring to execute. As a more intellectually rewarding alternative, consider deploying elastic scaling solution, or even fully-fledged autonomic resource optimisation. Infrastructure-as-code can help make it easier to keep tabs on what’s being run, and ephemeral leasing can help hurry the release of systems which aren’t actually needed anymore.

Vampire code and zombie servers are, fundamentally, wasteful. They use resources and have no value. That waste costs us money, but it also threatens the planet. Why not organise a few trick-or-treating expeditions to hunt down those under-utilized servers and useless lines of code?

Productivity is Messing Around and Having Fun

Thu, 10 Oct 2024 00:00:00 GMT

What is the happy path, for developers? It’s less boredom, and more play.

Developer satisfaction, developer joy, and business results are strongly correlated. And yet - many of our jobs are frustrating, filled with friction, and free of joy. What’s going on? How do we fix that? Is annoying waste inevitable? Can developer performance be tuned? Are productivity measurements helping or hurting us? How do we deal with fear at work? Will AI take our jobs? Finally, how can you persuade management to invest in boredom?

Zero Waste, Radical Magic, and Italian Graft – Quarkus Efficiency Secrets

Thu, 10 Oct 2024 00:00:00 GMT

What makes software efficient? Is it how quickly code executes, or is it how quickly developers can use it to solve problems? Quarkus makes both people and hardware more efficient. That’s cool, but how does it work? Usually, we expect to trade-off developer experience against runtime efficiency. In this session, Holly will dive into some of the technical underpinnings of Quarkus’s efficiency. She’ll give advice for those using or considering Quarkus - Do native binaries run fastest? Are verbose programs more efficient? She’ll also share general principles and transferrable discoveries from the engineering team, such as knowing what you’re optimising for, avoiding cache pollution, and how to be static-but-dynamic.

How Quarkus makes your apps cheaper, greener, and happier

Tue, 08 Oct 2024 00:00:00 GMT

Java never went away, but it’s having a renaissance. After some initial teething pains on the cloud (too big, too slow, too hard to configure) a new breed of Java frameworks are changing everything. In this talk, Holly will introduce Quarkus and then delve into some of the advanced features you might not know about. She’ll cover native binaries, what makes Quarkus super-fast running on the JVM, TDD with Quarkus, and Quarkus’s carbon footprint. The talk includes some theory (what underpins Quarkus’s surprising speed?) and also live demos (does Quarkus really start faster than a light bulb? What does an integration testing flow with Quarkus look like?)

Panel – Where Is Software Development Going?

Thu, 03 Oct 2024 00:00:00 GMT

The Efficiency Paradox and How to Save Yourself and the World

Thu, 03 Oct 2024 00:00:00 GMT

So far, so good, but we need to be careful we don’t end up accidentally optimising the wrong things, and making stuff worse. 100% utilisation is not sustainable for either humans, or people. It’s not even very efficient (what!?). Holly gives a brief tour of the brain’s default mode network, the latest business research, and queueing theory, to show why we actually achieve more when we do a bit less.

NLJUG academy masterclass – Create Java-based AI applications with Quarkus and LangChain4j

Thu, 26 Sep 2024 00:00:00 GMT

Developer Joy == Developer Productivity (really!)

Tue, 17 Sep 2024 00:00:00 GMT

Ever been told “work is not a place to be happy?” It’s not true! As developers, our job is to be productive, and we’re most productive when we’re in flow, doing friction-free creation. The good news is that new technologies help us do more, with less effort. (And only some of them are LLMs.) These developer tools and frameworks make us productive, and they bring joy. In this talk, Holly will present a number of productivity tips for the lazy, happy, developer.

How Quarkus makes your apps cheaper, greener, and happier

Tue, 17 Sep 2024 00:00:00 GMT

How to create your own Quarkus extension

Mon, 12 Aug 2024 00:00:00 GMT

Holly Cummins discusses the basics, tips and tricks of creating your own Quarkus extensions.

Using yq in GitHub Actions

Fri, 09 Aug 2024 00:00:00 GMT

TLDR: GitHub runners come with yq pre-installed. There’s no need to install it, or use a third-party action.

A problem of modern search engines is that it’s much easier to index the presence of something than the absence of it. (LLMs have this problem too, which is why ChatGPT will sometimes misunderstand a programming model and generate pages of unnecessary boilerplate.)

I recently used yq to alphabetise an array in a yaml document. I wanted to run my script after every commit to a GitHub repo, but I didn’t know if yq would be installed on the GitHub runners. So I googled “yq github actions,” as one does*.

The result was lots of marketplace actions for installing it.

There was a featured snippet for “is yq available on GitHub actions,” which directed me to a marketplace installer. The yq project itself had a marketplace installer. Clearly, I needed to install it before using it. Right?

I wrote my workflow, using the action from the yq project itself, and sent it off to review. My colleague George Gastaldi looked at what I’d done, and pointed out yq was available on the runners. This matters, because we try and limit our use of external, ‘non-official’ actions, for supply chain security reasons.

So I searched again to confirm, and … still found very little. To actually confirm, I had to merge and experiment. And, indeed, the GitHub runners do come with yq pre-installed. They’ve had yq since 2021.

This is the workflow step which alphabetises an antora playbook:

      - name: Alphabetise
        run: |
          yq '.content.sources |= sort_by(.url)' antora-playbook.yml > antora-playbook-sorted.yml
          mv antora-playbook-sorted.yml antora-playbook.yml

I’m hoping this blog can help the next person who might be trying to figure out how to use yq with GitHub Actions. For everyone else, please appreciate my attempt to illustrate ‘yaml,’ by combining a yak and lots of marks and language.

In hindsight, I perhaps should have been more precise and searched for “yq github runner,” but even if I had, the results are pretty much the same.

Productivity is Messing Around and Having Fun

Tue, 25 Jun 2024 00:00:00 GMT

What is the happy path, for developers? It’s less boredom, and more play.

Quarkus Efficiency Secrets

Thu, 20 Jun 2024 00:00:00 GMT

Is efficiency a good thing?

Tue, 11 Jun 2024 00:00:00 GMT

We live in a golden age of efficiency. Our modern cloud native applications have tiny footprints. Our infrastructure is now code. Everything is automated, even creative endeavours like writing and artwork. And yet. Despite all the optimisation, despite all the automation, many of us feel unproductive - and it’s worse than that. Busywork hasn’t gone away, our systems are stretched so thin any disturbance destabilises them, our people are burning out, and IT consumes more resources than ever before. What’s going on? Was efficiency the wrong goal, or are we just doing it wrong?

Faster, greener, and happier- why Quarkus should be your next tech stack

Mon, 10 Jun 2024 00:00:00 GMT

Productivity is Messing Around and Having Fun

Thu, 09 May 2024 00:00:00 GMT

What is the happy path, for developers? It’s less boredom, and more play.

Six things we learned implementing Rockstar on Quarkus

Wed, 08 May 2024 00:00:00 GMT

Let’s run Rockstar programs on Quarkus! What could possibly go wrong?

Rockstar is an example of an “esoteric language,” designed to be interesting rather than intuitive, efficient or especially functional. Rockstar’s interesting feature is that its programs use the lyrical conventions of eighties rock ballads. Rockstar has been implemented in many languages, but not as a JVM language. This was clearly (clearly!) a gap that needed fixing, so Holly and Hanno have stepped in to make sure us JVM folks aren’t missing out. As a bonus, because “Bon Jova” is a JVM language, it can take advantage of Quarkus-y goodness. Along the way, a lot was learned about eighties music, classloaders, parsing, bytecode manipulation, and the important relationship between language style, syntax, and semantics.

There will be live coding, live singing and live guitar!

Java Annoted Featured Content – The Long Version

Wed, 01 May 2024 00:00:00 GMT

I’m a JetBrains Community Contributor, and I was honoured to be invited to provide featured content for the Java Annotated newsletter. A week before it was due, I had a blank page. And then the day before it was due, I had a blank page. The day it was due, I had been thinking really hard, and had … a blank page with a few scraps of links. So I continued working on it. At some point, near the end of the day it was due, I reached out to the JetBrains team to ask how long it should be. “Oh, a paragraph or two” was the answer. “So, uh, eight paragraphs would be too much?” Yes. Yes, it seemed it would. Apparently, that’s what my own blog is for.

So, because I don’t like throwing things out, please enjoy the full, with-lots-of-extra-thoughts, version of the Java Annotated Monthly Featured content. Or, if you prefer the short version, it’s coming soon on the JetBrains site.

This past month, I’ve been enjoying the new full line code completion. There’s a lot of things to like about it; the feature is included in my existing IntelliJ IDEA Ultimate subscription, it runs locally, is privacy-friendly, is integrated into my existing tools, and it solves a real problem (typing is boring).

We’re in the midst of a full-on AI hype cycle at the moment. Last summer, Gartner announced that we had reached the peak of inflated expectations. Right now, many of us are flip-flopping back and forth between excitement and cynicism as we slide back down the hype curve. The potential of these tools is enormous, but so many of them promise a lot, and deliver … errors. And, if you look at the actual cost of training and running the larger models, it’s eye-wateringly, jaw-droppingly, horrifyingly, enormous.

There’s another problem with AI, which affects me personally. AI development all about Python, and I just don’t do Python. I’ve been a Java developer my whole career, and I quite like the idea of keeping it that way. If Python is the language of AI, does that mean us Java developers can ignore AI until we reach the trough of disillusionment, and after that don’t need to worry about it any more? Does it mean us Java developers are excluded from AI, whether we like it or not?

No. AI isn’t going away, but we’re starting to get a better understanding of where it’s useful and where it isn’t. Molly White, of web3isgoinggreat fame, has just published a fantastic piece called “AI Isn’t Useless”. I also wrote on my blog last month about the big question of whether AI will take developer jobs.

And AI is opening up to Java developers. This isn’t just AI-powered Java coding assistants; us Java developers can now create AI tools. LangChain4j simplifies integrating machine learning into Java applications. The Quarkus team have applied the Quarkus principle of developer joy to their Langchain4j integration, so it’s really nice. Last month, they released an Easy RAG extension, which allows you to set up a Retrieval Augmented Generation (RAG) application with just a single piece of configuration. You tell the easy-rag extension what directory your documents (PDF, Word files, etc) are in, and it will ingest them, decode them, and use them to teach itself. This makes it way more useful than a generic LLM, because it means the model has knowledge of the domain-specific things that you care about. I chat to some of my Quarkus colleagues about the Quarkus + LangChain4j in the latest episode of Quarkus Insights.

Another course-correction that I’m excited about is the revival of symbolic reasoning (that is, rules). We’re going to start talking more about combining large language models with symbolic reasoning. Rules engines are cheap to execute, easier to reason about, and reassuringly correct. Mario Fusco did a talk at Devoxx Greece about how LLMs and symbolic reasoning are complementary. The video isn’t out yet, although it might have appeared on the Devoxx YouTube channel by the time you read this. In the meantime, he’s prepared several sample applications, including an airline chatbot. He’s also got nice explanation of why symbolic reasoning is important in the readme. I think we’ll be seeing more of this kind of hybrid approach in the future, as our industry matures in how we use LLMs.

Being able to leverage AI from Java is important, because Java is getting better and better. One JVM improvement I’m excited about is project Leyden. Leyden isn’t news, but we’re seeing a nice stream of news and JEPs. Per-Åke Minborg from Oracle gave a nice Leyden updateat this year’s Jfokus. Both the slides and video are now available.

Leyden is part of a general movement to make Java start faster and run leaner. Historically, JVMs were optimised for dynamism. You could completely change the behaviour and dependencies of your application without stopping it. This is very clever engineering, but dynamism has a cost, and a highly dynamic runtime in a container is pointless. Even outside a container, realistically, most of us would (and should!) prefer to manage production updates through a CI/CD pipeline rather than patches to a live server. The good news is that Java is evolving, to reduce repeated work and unecessary classloading. Leyden is part of a bigger (and exciting!) trend of technologies to make Java start faster and run leaner, which also includes GraalVM, CRaC, and Quarkus.

Zero Waste, Radical Magic, and Italian Graft – Quarkus Efficiency Secrets

Tue, 09 Apr 2024 00:00:00 GMT

What is cloud native and how can it generate business value?

Tue, 02 Apr 2024 00:00:00 GMT

Knowing what cloud native is and where it benefits enterprise operations is essential for competitive cloud development

Will Generative AI Kill Developer Jobs?

Thu, 21 Mar 2024 00:00:00 GMT

Large language models are astonishingly good at producing large volumes of code quickly. That’s also, sort of, what I do for a living, except for the “quickly” part. And the “large volumes” part. So, will generative AI eliminate us developers?

Spoiler alert:

No.

I firmly believe developers are not going to be replaced by these models. Some industry figures are saying the opposite, quite loudly, so this isn’t an easy one to call. I’ll admit, some of the signs don’t look so good for us developers.

It hurts my professional pride to admit it, but LLMs are much better than developers at producing large quantities of code in a short time. And where a machine can do a job better, in general, job markets will shift and eliminate the human jobs. History is littered with jobs whose practitioners thought they were special and unique and un-disruptable, until … their jobs disappeared.

And, overall, I’m ok with that. These shifts can be painful at the time; no one likes losing their livelihood, much less losing their whole industry. If other opportunities don’t become available, it’s devastating for the people affected. It’s super-important, and humane, and fair, that businesses and governments help those who are economically displaced get back to meaningful employment. And we need to keep an eye on where the money is flowing when people are replaced by machines.

But even though job loss hurts, the alternative is worse.

I wouldn’t want me, or anyone else, to be spending time and effort doing a job that a machine could do. That’s just pointless busywork. Busywork doesn’t help the individuals involved, and it definitely doesn’t help us as a species. Automation has been responsible for a huge rise in global living standards over the past two hundred years.

A short, selective, history of automation.

Many things we used to do by hand, such as washing clothes or weaving fabric, are now done by machines. (The economic disruption caused by automated fabric weaving led to a great deal of social disruption, and gave us the words ‘sabotage’ and ‘Luddite’.)

Jobs have disappeared, and technology was to blame. Automation killed the job of the washerwoman, and the stockinger, and also the knocker-up. But wait, what on earth is a knocker-up?

Do you remember alarm clocks? They were the things we used to use before we all used our phones as alarms. They were mechanical, and are themselves a relatively recent invention. Before people had alarms, how did they get to work at the right time?

Often, “the right time’ was the time that they got to work. Cows and carrots and fields are pretty forgiving of tardiness. It was only when people starting working together, in shifts, in expensive factories, that clock-time became important.

This new need for punctuality spawned a new job: the knocker-up.

The knocker-up was a human alarm clock. Knockers-up would visit their clients and wake them up, either by tapping on the window with a stick, or by shooting their window with pea-shooters.

(If you’re reading this and are confused or horrified, the important context you’re missing is that ‘knocking up’ means ‘to wake someone up’ in UK English. In North American English ‘knocking up’ means ‘to impregnate’, but that kind of service was definitely outside the job description here.)

Knocking-up was a job created by the changing technology of the industrial revolution, but changing technology also eliminated the profession. As more people got mechanical alarm clocks, the need for the knocker-up faded. The need didn’t fade as fast as you might think, though; professional knocking-up lasted to the 1940s and 1950s, and in some parts of England it hung around until the 1970s.

Even though it survived a surprisingly long time, it’s easy to see why the knocking-up industry was ultimately doomed:

A convenient technological alternative was available
The pool of the population who need waking up is limited; only people working outside the home on a strict schedule wanted the service
Each client needed the service a maximum of once a day; no matter how much marketing a knocker-up did, no matter how great the knocking-up service, people didn’t need re-waking once they were already awake

A cheap alternative, and finite demand. Clearly, knocking-up couldn’t last.

Why software development is different (really!)

So what’s different for developers? Almost everything. Although software-writing automation is (sort of) available, it is not a convenient replacement for human developers. That’s true now, and it’s likely to stay true. Crucially, too, the world’s demand for software is apparently infinite, or would be if there weren’t budget constraints. That means demand for software developers is large, and growing.

The role of the developer - writing code is only a small part

Why can’t Copilot, or ChatGPT, or Devin, or other generative AI equivalents, do the job of a software developer? These tools are very, very, good at copying and pasting code from the internet. Software developers are also very good at copying and pasting code from the internet, but these tools are even better.

But the job of a software developer is not to copy and paste from the internet.

That’s part of the job, but only part.

The role of the knocker-up is fairly well-defined. It’s … to wake people up. This is different from the role of, say, a hairdresser, who is partly there to cut hair, and partly there to act as a therapist.

The role of a software developer is similarly blended, except the computer is the one on the therapy couch. Even before we start writing code, our job is to understand what problem is being solved, understand why the original problem statement was flawed, re-define the problem, and explore risks. Then, after we’ve written (or copy-and-pasted) the code, our job is to get code from the internet all the way into production, debug the code, debug the code again, understand the computer’s deepest secrets, try and stop the computer system from going completely mad, work with colleagues to integrate incompatible systems, manage stakeholders, manage other stakeholders who are quite obstreperous, debug the code again since it’s doing something in production which no one has seen before, and then re-define the problem again.

Fundamentally, our job as software developers is to solve problems which have never been solved before. I mean, if the problem had been solved already, why would we want to solve it again? Let’s just get a computer to do that for us.

Demand is infinite

In my experience, software creates the need for … more software. Think, for example, of Jenkins, or Tekton. Or Kubernetes. Or, in fact, the whole of the CNCF landscape. These are complex pieces of software which only came into existence because other software existed. Think of ad-blockers; their whole existence is a response to other software.

Software multiplies. Software snowballs.

The world doesn’t just demand software. It demands:

software to do stuff
software to do more stuff
software to bypass advertisements in the software
software to bypass the bypass software so ads still show (the ad-blocker arms race is creating a lot of jobs)
software to monitor the software
software to debug the software
software to manage the software
software to write the software
software to manage the software that writes the software
software to try and avoid skynet
software to tell people about software
software to work out the meaning of life
software to gamify software

Past productivity gains did not eliminate developer jobs

My assertion that the development for software developers will increase isn’t just a reassuring wish. We can look at the impact of earlier changes in our industry. Each one made software development easier (in some way), made it accessible to a wider group, and made developers more productive. And yet following each productivity gain, demand for developers increased.

Programming a computer used to be hard. Before we had IDEs and MacBooks, we had punch cards and vacuum tubes.

In 1947, Kathleen Booth created the first assembly language, which made programming more efficient. In 1956 John Backus and an IBM team developed the first high-level language, FORTRAN, which made programming more efficient. By the mid-1960s, higher level languages were enriched with reusable libraries, which made programming more efficient. Somewhere in the 1980s we got frameworks, which made programming more efficient. In the 1990s we got mainstream garbage collection, which made programming more efficient. In the 2000s we got and smart completion, which made programming more efficient.

So did each productivity improvement reduce the number of developers working in the industry? Er, not exactly. In the 1950s, there were a few hundred developers in the world. In 1970, there were around half a million. In 2024, there are around 28 million developers in the world. It’s Jevons paradox, but for people. And exponential.

Source for earlier data, source for recent data

As Gergely Orosz put it this week, writing about the same question, the goal of COBOL “was to allow business people with no programming background to use it. In the end, COBOL didn’t remove the need for developers: instead, it created demand for COBOL developers.”

But our jobs will change

Even if AI is unlikely to take our jobs, it doesn’t mean it won’t change them. When a productivity improvement is available, it is expected that people use it. None of us, not even the most curmudgeonly technology-resistant developer, program like it’s 1947. If we did, we wouldn’t have achieved cat videos and Baldur’s Gate and online shopping. None of us are using punch cards, or GOTO statements. We use IDEs, and Testcontainers, and Quarkus. We wouldn’t write a greenfield application in COBOL, except as a dare. Some hardcore folks use assembler, but only in contexts where being close to the metal is worth the productivity penalty.

I have some questions about how much the current batch of tools actually improve productivity, but that’s a separate topic. I think we can safely assume that some of the quality issues with generative AI will be ironed out. These tools will become embedded in our workflow, just as IDEs and compilers have. They will take away some drudgery and toil, but they will not take the jobs. The computer assistants are coming, but the people aren’t going away.

The future of development: Are our jobs getting harder or easier?

Fri, 15 Mar 2024 00:00:00 GMT

LCC 307 - Interview sur la passion, la créativité et le fun sur le long terme avec Holly Cummins et Mark Jane

Sun, 03 Mar 2024 00:00:00 GMT

Enregistré le 27 février 2024

Is Efficiency A Good Thing? Part II: All the Things That Can Go Wrong

Sat, 02 Mar 2024 00:00:00 GMT

In part I of this blog, I discussed what efficiency even is, and explored the question of whether we’re any good at it. Although in some ways we’re getting more efficient, many of our organisations and software systems are still shockingly inefficient.

I get intensely annoyed by waste. All of us should take it as a personal mission to eliminate waste. As techies, we have a brilliant opportunity to do this by streamlining processes and optimising software. We should also go further; sometimes, instead of being streamlined, a process should just be eliminated. The management consultant Peter Drucker once said “there’s nothing quite so useless as doing with great efficiency that which should not be done at all.”

My team used to have a regular meeting in which we set the priority of incoming defects. For every defect, we would discuss the potential impact, and then set the priority to “medium”. Every. Single. Time. (We had another field, Severity, which had more meaningful variation.) Eventually, someone suggested that perhaps we could save some effort by writing a shell script which automated setting the priority to “medium”.

This was a good idea, but there was a better one. If the field had so little value that we could manage it with a shell script, why did we even have the field? Why not just remove it from the tooling, and eliminate some clutter in our UI?

I sometimes have the same thought about some of the code and verbiage generated by gen AI tools. If the code, or comment, or words, are so predictable that ChatGPT can write it, do we really want that code? AI is good at writing boilerplate, but the best thing to do with boilerplate is eliminate it. A GitHub study found that Copilot users accepted 30% of its suggestions, and Copilot produced 40% of the code. These figures seem impressive on the surface, but the more you unpack them, the less good they are. First of all, a 30% success rate means that 70% of the time, Copilot suggested the wrong thing. But what about that 40% of the code that was written by the AI? I suspect such a codebase will have a low information density – lots of boilerplate, lots of code comments ‘documenting’ parameters whose purpose is self-evident from the name, and so on.

I’ve certainly seen this kind of bloat in my own experiments with ChatGPT. When I asked ChatGPT to write me a Quarkus PanacheEntity, I was initally delighted with how much code it produced. “Wow,” I thought, “it would have taken me ages to write that much code!” But when I looked closer at the code, I realised I wouldn’t have written that much code, because the code didn’t need to be there.

70% of what the AI produced was waste. It used a bad, outdated, design where it explicitly filled in getters and setters that would normally be generated by the Panache framework. One of the design features of Quarkus is how it eliminates boilerplate, but ChatGPT had carefully put all the boilerplate right back in.

This kind of pointless code may seem unnecessarily flabby, but otherwise harmless. But superfluous code and comments is noise. It inteferes with our ability to understand code and distracts us from the important parts. It’s also a maintenance liability. For example, generic, value-free, comments can easily go out of sync with the code. When that happens, they’re no longer value-free, they’re value-negative.

Our industry learned, long ago, that measuring developers by the lines of code they write is a bad idea. We now need to learn not to measure our AIs by that criteria.

Measuring productivity

All organisations want to be data-driven. What does productivity data look like? How do you measure productivity, if it’s not by lines of code? Some people joke that you should measure junior developers by how much code they write, and senior developers by how much code they delete. This is a nice way of thinking about the difference in roles, but it’s not a proper productivity metric.

You get what you measure, so make sure what you measure is the thing you really care about. This is harder than it sounds, because often the thing we really care about is hard to measure. For example, an important role of senior developers is to produce other senior developers, but how do you measure the subtle chains of influence involved in that process?

Often, we resort to proxy measurements. This can be dangerous, if we’re unwise in our choice of proxies. I saw a comically awful social media post recently, in which a VC boasted that his founders were measuring their sleep using a Whoop group. The goal wasn’t for the founders to sleep more so they could make good decisions, it was for the founders to sleep less so they could demonstrate … commitment, or drive, or something equally nebulous. The founders were averaging 5.5 hours sleep a night, and this was judged to be a great result. This is not a great result. The effects of sleep deprivation are similar to the effects of alcohol; being tired makes people careless and stupid and causes all sorts of bad decisions and workplace-related accidents.

Being sleep-deprived is an illusion of efficiency, not real efficiency. Using AI to generate reams of value-free code is an illusion of efficiency, not real efficiency.

Why inefficiency is efficient

Sometimes, real efficiency looks like inefficiency. This is a complicated stuff! No wonder we get it wrong.

There’s an enormous amount of research that shows fun in the workplace is good for business. If people are happy at work they work harder, take less sick leave, and are more productive. In 2014, the first DORA report established that job satisfaction is the number one predictor of performance against organizational goals.

At an individual level, the Harvard Business Review found your brain in a positive state is 31% more productive than your brain when it’s neutral or stressed. How can such a positive state of mind be achieved? Well, cat videos are part of the solution. A study from the University of Warwick found that people performed 12% better on a test if they’d just watched a comedy video. Most managers would be delighted if you told them you knew how to achieve a 12% performance improvement … at least, until you mention it involves comedy videos.

Not only does having fun at work improve productivity, staring into space can improve productivity. Doing nothing creates the ideal conditions for creativity and problem-solving. For our industry (lucky us!), creativity and problem solving are key elements of our productivity. That means doing nothing is a key part of our job.

What underpins the productivity of idleness? The default mode network is a pattern of brain activity which kicks into life when the rest of the brain goes into an idle state. For example, taking a shower or going for a run can trigger the default mode network. The default mode network is associated with mental time travelling, creativity and problem-solving, so triggering it is a good thing.

It’s not only fluffy-wuffy psychology which says idle time is important for productivity. Un-fluffy, un-wuffy, mathematics confirms it. Queueing theory is the science of how and when works get done.

It could be a computer doing the work, or a person; either way, the mathematics are the same. A getting-stuff-done process is modelled as

an arrival process (new work coming into the system)
a queue (work requests waiting to get sorted out)
servers (people, or threads, or machines, or whatever is doing the work)
completed work (what gets spat out at the end of the process)

The arrival process is usually assumed to be a Poisson distribution; that is, random, but distributed around some average arrival rate. If server capacity is too low, a queue builds up, and wait times are high. If server capacity is high, the queue will be mostly empty, and some servers will be idle. Is this an efficient situation, or is it inefficient? It depends what you’re measuring. Requests are handled quickly (efficient!), but there’s a lot of wasted capacity (inefficient!).

What makes this trade-off particularly interesting is that it’s asymmetric. Because requests come in somewhat randomly, there will be times when requests bunch up a bit, and queues build up. But what happens when requests are unusually sparse? The queue can’t build down below zero.

Because a queue length can’t be negative, busy times hurt the system more than quiet times help it. You can see this effect on a plot lead time as a function of utilisation (how much of the time servers are busy). With a Poisson distribution of arrival times, going from 80% utilization to 90% utilization doubles wait times. If the servers are busy 100% of the time, lead times are infinite. There has to be some slack in the system, or it collapses.

Lead times (purple line) and resource cost (green line) as a function of utilisation. Source: http://brodzinski.com/2015/01/slack-time-value.html

This is the reason train schedules have some slack in them. Trains usually travel a bit below their maximum speed, or pause for longer than strictly necessary at each station. Otherwise, any minor delay could perturb the system unrecoverably.

On the other hand, too much slack is unaffordable. The green line in the plot above shows the cost of the idle capacity; at low utilisation, it’s big. Provisioning a dedicated pool of hundreds of build machines ensures build wait times are short, but it’s ruinously expensive to have so much idle capacity, and no business would do it. I get my best ideas in the shower, but it wouldn’t work if I spent seven hours a day in the shower, and only one hour at my desk. The management skill is to balance the competing inefficiencies, and come up with something that just about works.

Is Efficiency A Good Thing? Part I: What is Efficiency, and Are We Any Good At It?

Fri, 01 Mar 2024 00:00:00 GMT

This blog is based on a keynote I presented at Tech.Rocks Summit 2023. I’m grateful to the organisers for suggesting the topic.

Is efficiency a good thing? On the face of it, the question seems absurd. Of course efficiency is a good thing. None of aspire to be inefficient. Most of us strive to be as efficient as possible, and to make our organisations equally efficient.

And yet. Our obsession with efficiency is actually relatively recent. If you search “what is efficiency?,” Google will suggest you might also like to search for “when was efficiency invented?“.

Forms of efficiency

People first started thinking about efficiency in the 1700s. At the time, it was in a quite specific context: machines. Steam engines were machines which could convert heat energy into mechanical energy – that is, movement. That mechanical energy drove machines which did useful things, such as pumping water or moving a vehicle. But the conversion was not particularly effective; the vast majority of the heat energy was just … lost. Scientists and engineers worked hard to make these engines more efficient, so that less coal required for each unit of useful work.

As communication and transport technology advanced, the scale of administrative structures increased. By the late 1800s, the world had large, empire-scale, governments, and large, multi-country corporations. These large structures had a natural tendency to descend into chaos, if there was too little process, or bureaucracy, if there was too much process. Efficiency shifted to become a management concern. Researchers started exploring how to optimise the interaction of human beings at scale, so that each process required the minimum amount of time and money to produce meaningful value.

Early in the twentieth century, this optimisation effort was extended to factories. Factories take in things, which have some value, and produce other things, which hopefully have more value. How should the conversion of things to other things be done, so as to maximise the increase in value? Time-and-motion studies aimed to eliminate wasted effort in manufacturing processes.

In the nineteen sixties, a new form of efficiency became important: software efficiency. Given enough hardware, and electricity, and time, software could convert questions into answers. How do we reduce the amount of resources required, and perform calculations as quickly as possible?

We’re good at efficiency

After two hundred years of effort, we’re getting pretty good at efficiency. By the time they were phased out in the 1950s, steam engines were twenty five times more efficient than they’d been at the start.

source: http://geosci.uchicago.edu/~moyer/GEOS24705/2016/Assignments/PS7.pdf

You’ve probably heard of Moore’s law, which predicts that the number of transistors on a chip doubles every two years. Koomey’s law is a related observation: the energy efficiency of computers doubles roughly every eighteen months. This is brilliant. (If it wasn’t for Koomey’s law, we wouldn’t be able to afford Moore’s law!)

Software efficiency also continues to improve. I’m a Java person, and it’s noticeable that each new release of the JVM goes faster than the previous release. The frameworks we build on top of the JVM have also become astonishingly efficient. In 2010, if you wanted to launch your code on an application server, you had enough time to make a cup of tea while it was starting.

By 2022, GraalVM, in combination with Quarkus, could start an application in 15 ms, faster than an LED lightbulb. (Did you know LED lightbulbs take some time to start up? I didn’t, until I started comparing their start times to Quarkus native start times.) Even when running as a vanilla JVM application, without GraalVM’s native compilation, a Quarkus REST application starts in under a second.

A lot of this performance improvement has been driven by the move to cloud. Cloud applications tend to be at least a bit ephemeral, so startup time matters. Footprint also matters, because there’s a financial incentive to try and pack many cloud applications onto a single physical machine.

Our systems are getting quicker, and we’ve also greatly reduced toil and repetitive tasks. Infrastructure-as-code and GitOps has allowed us to automate our infrastructure. AI tools have automated us. Dall-E can produce much better kangaroos than I can draw, in just a few seconds. ChatGPT can write an coherent essay exploring the benefits and risks of efficiency in seconds, when it takes me days of effort to write the essay you’re reading now. That’s not even counting the thinking-time. I started thinking about this topic in March 2023, and it was only in December 2023, nine months later, that I had a draft essay. Then it took me a further three months to finish off the draft. That’s an elapsed time of one year, or 3,153,600 times slower than ChatGPT’s elapsed time.

The worst part? ChatGPT came to broadly the same conclusions as me. When I saw that, I was pretty deflated. ChatGPT even included some of my points that I’d thought were all clever and novel.

No, we’re really bad at efficiency

So is efficiency a solved problem? Not quite. In fact, we’re astonishingly terrible at all forms of efficiency.

Let’s start by considering process efficiency. Many years ago, pre-cloud, some of my colleagues sold a nifty provisioning system that would allow people to do self-service requests for virtualised infrastructure. They promised the client instances could be created in ten minutes, which at the time was impressive. (Remember, this was pre-cloud.)

To their dismay, the client came back with a complaint. They weren’t getting new instances in ten minutes. The spin-up wasn’t an hour, or ten hours. It wasn’t even ten days. It was taking the system three months to provision a new instance. My colleagues were puzzled. How could this be? On investigation, it turned out the problem wasn’t technology, it was people. The client had added some guard rails and governance to the system. And then a few extra guard rails. What with one guard rail and another, there was an 84-step pre-approval process for each instance.

So, we all know people can be inefficient. But surely our software is efficient, right? Well, not necessarily. In 2017, LinkedIn did a research trip to Nashik, a small city about 100 miles from Mumbai.

The team wanted to see how people were using one of their new features. The answer was, they weren’t. The site failed to load, because even in the city, connectivity was too poor. Many modern web sites have similar problems. The sites have so much bloat, they’re actually useless in large parts of the world.

Waste and bloat is everywhere. A quarter of our servers are zombies - that is, they haven’t delivered any computing services for six months. A further quarter are utilised less than five percent of the time. The average server in our industry has utilisation between 12 and 18%. That’s just not good enough. It’s financially wasteful and environmentally destructive. All computer systems have overheads, so even at idle, that server still consumes significant electricity. And then there’s the physical hardware. Manufacturing hardware takes raw materials, and water, and energy. There’s only a return on that investment if the hardware is used.

All of this inefficiency is unfortunate, but accidental. We’d perhaps like to get rid of it, but we haven’t really put our minds to the problem.

But sometimes, we cling to inefficiency. Deliberate inefficiency is a common organisational anti-pattern. Team members won’t report an inefficiency, because eliminating the inefficiency might mean eliminating their colleagues. Inefficiency creates employment. Dysfunctional employment, but employment.

Several software solution vendors have told me similar stories about the challenges they have selling their product. Their nifty technologies make teams more efficient, and allow them to do more with less. But it turns out, the managers of those teams don’t want to do more with less. They want to do more with more! We measure the status of leaders by how big their budget is, how big their team is. Managing a team of hundred is more prestigious than managing a team of twenty-people-plus-some-great-software. So why would you buy something that potentially reduced your status?

Are we doomed to inefficiency, then? Well, not necessarily. We can get rid of inefficiency, and we can do it without hurting people. In part II I explore some good ways of eliminating inefficiency, and also some bad-and-wrong ways.

Six things we learned implementing Rockstar on Quarkus

Tue, 06 Feb 2024 00:00:00 GMT

Let’s run Rockstar programs on Quarkus! What could possibly go wrong?

There will be live coding, live singing and live guitar!

The future of development: Are our jobs getting harder or easier?

Fri, 26 Jan 2024 00:00:00 GMT

Tradeoffs, Bad Science, and Polar Bears – The World of Java Optimisation

Thu, 25 Jan 2024 00:00:00 GMT

Welcome to the Java optimisation jungle. Why can’t we “just make it go faster”? It turns out, in most cases, we need to first work out “faster for whom?” and “why do we want to go faster?” and “what even is faster?” This talk introduces the basic principles of optimisation, before bouncing through the pitfalls of optimisation; why the exact same techniques which make Quarkus rocket-fast used to be a terrible idea fifteen years ago, why fast benchmarks make for slow programs, why project loom may not be the speedup you’re hoping for, and why even though it can be easy to get wrong, optimisation really really matters. Along the way we’ll talk about measuring things, bad advice, garbage collection, and climate change.

Is efficiency a good thing?

Fri, 08 Dec 2023 00:00:00 GMT

The AI skills gap is becoming incredibly troubling for companies everywhere

Thu, 07 Dec 2023 00:00:00 GMT

The solution isn’t so clear, report claims

Cloud Native Users Struggle to Achieve Benefits, Report Says

Tue, 28 Nov 2023 00:00:00 GMT

Of those organizations that have “gone cloud native,” 95% said that challenges are keeping them from seeing the full benefits, in a new survey by Foundry for UST.

Quarkus Unveiled: Efficiency & Green Impact • Holly Cummins & Charles Humble • GOTO 2023

Tue, 21 Nov 2023 00:00:00 GMT

This interview was recorded at GOTO Aarhus for GOTO Unscripted. #GOTOcon #GOTOunscripted #GOTOaar http://gotopia.tech Read the full transcription of this inter…

Spécial Tech.Rocks Summit 2023 - Holly CUMMINS (Red Hat) et Philippe ENSARGUET (Orange) - #S05EP31

Sun, 12 Nov 2023 00:00:00 GMT

🎤 Dans cette nouvelle mini-série dédiée à notre Summit qui se tient les 7 et 8 décembre prochains en hybride, nous avons aujourd’hui le plaisir de recevoir une de nos speakers 2023 : Holly CUMMINS, Senior Principal Software Engineer chez Red Hat, au micro de Philippe ENSARGUET, VP of Software Engineering chez Orange.

Holly revient dans ce podcast sur les enjeux de développer des applications plus vertes, mais également sur ce que Holly appelle le SwitchLightOps, la capacité à éteindre nos applications quand nous ne les utilisons pas.

Holly établit un lien entre l’efficacité et la consommation de nos solutions. Plus une application est efficace, moins elle consomme. De la même manière, plus elle est efficace, moins elle coûte cher en i

The necessary death of software systems

Tue, 31 Oct 2023 00:00:00 GMT

Sometimes, death is a feature, not a defect

It’s Hallowe’en! Let’s talk about mortality, death gods, and zombies.

I know, I know, you studied computer science instead of biology precisely so you didn’t have to deal with dead things. Most of us who work with software spend a lot of effort trying to keep our applications alive. If you’re an SRE, keeping systems alive is basically your whole job description.

Ryuk and the god of death

But you know what? Death is a necessary fact of life, especially in the land of bits and bytes. I realised this when I was trying to sort out some compatibility issues between Quarkus, Podman, and Testcontainers. In order to get Quarkus Dev Services (which use Testcontainers under the covers) to work with Podman on my Mac, I had to adjust my environment variables to disable something called Ryuk:

export TESTCONTAINERS_RYUK_DISABLED=true # this is a bad idea

With this, everything more or less worked … but it turned out it wasn’t the right way to solve my problem. Why? What was Ryuk, anyway? This comes back to the question about what Testcontainers is doing, or ‘why would I useTestcontainers rather than just starting containers directly?’ (If you’re using Quarkus, the answer is that you actually don’t use Testcontainers directly, Quarkus uses Testcontainers for you to transparently start all your dependencies before you or your tests need them. But let’s not worry about that for now.)

Testcontainers does two really useful things; it has libraries in all the popular languages to interact with a container runtime. That means instead of having to fuss with Runtime.exec or exec.Run, you can just call a nice API to start your container. But the other useful thing Testcontainers does is stop your containers. No more discovering a container still hogging your RAM days after you last used it, just because you forgot to shut it down.

How does Testcontainers shut down abandoned containers? This is where Ryuk comes in. The Ryuk service is a specialised container whose job is to clean up other containers. That is, it’s a container-killer. A helpful, appreciated, container-killer. And the name? Ryuk is named after Ryuk in the Death Note manga series. Ryuk is a shinigami, or death god. Although Death Note’s Ryuk is capricious and self-interested, in Japanese folk culture shinigami are often presented as helpers that lead people towards the world of the dead, a bit like a friendlier version of the grim reaper.

So Ryuk definitely isn’t something we want to disable. It’s garbage collection at the process level. No one wants stray containers living forever. The right way to configure Testcontainers for Podman, on Mac, is to edit ~/.testcontainers.properties and add the following line:

ryuk.container.privileged=true

What’s this doing? It’s making Ryuk, the death god, a bit more god-like. You will also need to run Podman in rootful mode, so that it has privileges to pass on to Ryuk:

podman machine set --rootful

Do we have evidence how useful stopping things actually is? We do.

The zombie menace

If a Testcontainers container doesn’t stop, it’s a minor irritation. It might consume a little bit of CPU and memory until your laptop is next rebooted, or in the worst cases, it might cause interference and weird results in subsequent test runs. But when “not stopping things” is multiplied across an entire industry, the consequences can be serious.

The Anthesis institute have done a series of studies to work out how widespread zombie servers are. They found that 25-30% of servers are “comatose” – that is, they haven’t done any useful work for six months. Here, “useful work” doesn’t mean “they weren’t just serving cat videos,” it means they haven’t delivered any information or computing services for six months or more. That’s a shockingly low bar, and yet 25% of systems still didn’t meet it.

What if we raise the bar slightly and look at servers where the utilisation was under 5% (which is still a pretty atrociously low bar)? These ‘under-utilised’ servers made up another 29% of the pool. If you combine the totally-comatose and mostly-comatose servers, that makes up two thirds of the servers.

Even systems that are in regular use are often only used for part of the time. In 2021, $26.9 billion was wasted on always-on cloud instances.

In other words, we’re paying for hardware, and electricity, for systems that do … nothing. Sustainable, it is not. What our industry needs is a nice friendly shinigami to take down these systems.

We’re starting to see shinigami which can shut systems down out of hours. For example, the DailyClean tool provides a friendly interface for scheduling ‘alive times’ for Kubernetes workloads.

Randomised rebirth

However, zombie-destruction is a harder problem to solve. If something is never being used, why did no one just turn it off? Often, these systems have been lost out of institutional memory, because a process changed and the old process was never properly retired, or a team was disbanded or a project stopped. Forgotten systems are, by definition, invisible.

Sometimes, people know the system is useless, but it’s too much trouble to shut the systems down. This is a particular problem in regulated industries, which are (understandably) averse to casually shutting down parts of their infrastructure. I heard a story of an outsourcing provider that shut down what they thought was an unused server. Shortly after, the phone rang with a very very angry client on the other end. The ‘unused system’ turned out to be the backbone of the client’s network infrastructure, just mislabelled. Avoiding this kind of awkward scenario is why some companies enforce heavy bureaucracy before letting any of their employees go anywhere near a power switch. In these organisations, the paperwork to prove a system is unused can take many months.

There is a better way. I mentioned that SRE is the business of keeping systems alive. One of the tools SREs use to do this is regularly killing systems. What? Really. Damaging infrastructure creates resiliency, and killing infrastructure creates immortality. It’s a bonkers paradox, but it works. Chaos engineering is the practice of arbitrarily shutting down parts of a system to exercise recovery processes. This is known as deploying chaos monkeys. In the ideal case, the system copes with the loss of a member by failing over or routing around the casualty. In the bad case, there is learning!

We can extend this to zombie-hunting. One way of detecting zombies is known as a “scream test” (very Halloween-y!). The system gets shut down, and then the team wait for screams. If there were none, it wasn’t being used. This methodology can be formalised as an eco-monkey. As long as everyone is confident a system can be brought back up (‘LightSwitchOps’), it can be subjected to a test-shut-down with relatively low risk. If that sounds a bit too scream-y for comfort, AIOps can identify under-utilised systems by monitoring traffic and hardware load levels.

Killing systems is important. It’s important to extend their life, paradoxically. It’s important to save resources, and save the planet. And it’s important so that our laptops don’t grind to a halt under the weight of two dozen unused databases. So this Halloween, let’s give some thanks to our techno-shinigami, and see if we can write some more.

I’m grateful to Ixchel Ruiz for telling me about the origin of the Ryuk name, and for the conversation that inspired this blog.

The Vroom Model, or Why Naming is The Hardest Problem In Computer Science

Mon, 30 Oct 2023 00:00:00 GMT

TL;DR

If you’re looking for the talk where I describe the Vroom Model, the best source is this GOTO Aarhus talk.

You might also like to browse everything I have about sustainability.

Background

I use Algolia in my site stack. It’s one of the more pointless parts of the site, because it doesn’t get used much. Every week Algolia mails me to tell me that no one has searched for anything.

Every now and then, someone actually does search for something, and I get all excited. Sometimes someone did a search but Algolia had no results to return, which is a shame. If someone took the time to search, I want to make sure my site helps them find what they need.

Twice, the failed search has been ‘vroom model.’ Since there’s not many searches, this makes ‘vroom model’ both the most popular search on my site, and the least successful search on my site. Oops. Why is ‘vroom model’ not finding anything? Two reasons:

I index the titles of externally hosted content such as talks, but not the contents. I should sort that out, but it’s a bit involved.
I am an idiot, and gave my model a un-searchable name. I didn’t actually call my model ‘vroom model,’ I called it ’ vrrrooooom model’. Or perhaps it was ‘vrooooom’ model. Or maybe ‘vrrooom model’? The number of ‘r’s and ‘o’s in ’ vrrrooooooom’ is totally non-deterministic. That looks cute on a slide, but is hopeless for trying to find it by text search. When I looked, I discovered that I’ve used several different spellings, even in my own writing. More evidence, if any were needed, that naming is the hardest problem in computer science.

How hard is naming? Well, it turns out there already is something called the Vroom Model (one ‘r’, two ‘o’s). It was invented in 1973 by Victor Vroom, working alongside Phillip Yetton and Arthur Jago. The original Vroom Model, or, to be more complete, Vroom-Yetton-Jago Normative Decision Model is about decision making, not about sustainability.

If you use Google search for “vrrroooom model,” it will search instead for “vroom model,” and show you Dr. Vroom’s model. If you force it to search for the original spelling, with all the r’s and o’s, it will show you a set of images which are barely safe for work. They involve cars. And bikinis.

Ok but what is Holly’s Vrrroooooooooom Model?

Let’s assume you’re interested in green software rather than sexy vehicles. What’s the vrrrooooom model, in that context? While we’re here, I’ll explain. Calculating carbon emissions is important, but hard. Sometimes it’s important to be accurate, but if you’re optimising, you don’t need a perfectly precise absolute number, you just need to be consistent in your measurements so that you know you’re going in the right direction. Sarah Bergman made this point when discussing the Green Software Foundation’s Software Carbon Intensity specification. Sometimes you’ll need to guess about the exact values of some constants when doing SCI calculations, but as long as the external value does stay the same, that’s enough to allow you to do data-driven optimisation.

But sometimes even that level of measurement is too much work. We all have competing priorities and too much to do. Does that mean we should just give up on carbon efficiency if we don’t have time for rigorous measurements? No way!

In some of my talks, I describe two useful heuristics for reducing carbon footprint. I should emphasise that these are heuristics, and they will never be as good as proper measurements. Remember, all models are wrong, but some are useful.

The Economic Model

The first model is the economic model, which I learned from Adrian Cockcroft. The proper name is “economic input-output life cycle assessment.” That’s a bit of a mouthful, which is why Adrian calls it the economic model. In the simplest terms, if you reduce your company’s cloud bill, or data centre bill, or electricity bill, you’re probably reducing your cloud footprint. This only works if you keep everything else the same; you can’t just shift consumption from one account to another and claim a win, or move to a cheap-but-polluting supplier and claim a win.

The Vrrrrrroooom Model

The second model is the vrrooom model. I made the name of this one up, which is where the failed-search-problem started! The vroom model says that energy consumption is (sort of, mostly) proportional to execution time. Again, this only works if you keep other things the same; it’s cheating to upgrade hardware or move to a massively parallel cloud execution model and then claim a win because your code is running faster. There will also be some workloads that break this model. For example, memory contributes to carbon footprint, so an increase in memory consumption could drive execution time down but energy consumption up.

For a quick visual to see how well the heuristic stacks up against reality, have a look at the results table in the internet-famous “Energy Efficiency Across Programming Languages” paper. There are some quite legitimate critiques of this paper. The measurements here were based on a series of micro-benchmarks, and should be taken with some salt. Nonetheless, I find the correlation between time and energy pretty striking. I’ve plotted the authors’ results, and you can see the line is pretty close to straight. C is fastest and uses the lowest-energy, Rust is very very close to matching C, and Java does pretty well in both execution speed and energy use. JavaScript is middle of the pack, but notice how TypeScript is far slower and more energy-hungry. PHP, Perl, and Python bring up the rear.

The relationship between energy and execution time for a series of benchmarks. Values are normalised against the results for C. Data source: Pereira et. al

Going beyond languages to frameworks, we see a similar pattern with Quarkus. Applications built on top of Quarkus tend to have a higher throughput, and they also have a lower energy consumption.

I think we can take a lot of encouragement from both the vroom model and the economic model. Everybody likes low bills and fast software. Cheap and speedy IT is its own reward, but the economic and vroom models tell us that it is also green. Result.

Expert Talk: The Current State of Software Engineering • Jez Humble & Holly Cummins • GOTO 2023

Tue, 24 Oct 2023 00:00:00 GMT

This interview was recorded at GOTO Aarhus for GOTO Unscripted. #GOTOcon #GOTOunscripted #GOTOaarhttp://gotopia.techRead the full transcription of this inter…

The future of development: Are our jobs getting harder or easier?

Sat, 21 Oct 2023 00:00:00 GMT

8 Talk Title Mistakes We've All Made

Thu, 12 Oct 2023 00:00:00 GMT

Why crafting a good presentation title is annoyingly important, and annoyingly hard

I hate choosing titles for my talks. Titles have to be good. They have to be really good. Titles are the difference between a full room and you speaking to a room containing two colleagues and someone who got lost looking for Venkat’s session. That’s a lot of pressure for six (or so) words.

Not only do titles need to be good, we have choose them at the time when we’re least equipped to do so. By definition, conferences and meetups need to publish talk titles early. But because it’s only sensible to hold off writing my talk until after it’s been accepted by a conference, I don’t entirely know what the talk will be about when I choose the title. Even in the good case, I only know what the talk’s about once I’ve finished all the slides. In the bad case, I have to deliver a talk a few times before I properly figure out what my point even is, but that’s a topic for another post.

Just because I’m bad at talk titles myself doesn’t mean I don’t notice other people’s mistakes. So here’s my “do as I say, not as I do” guide to choosing a good title.

Your title has to work hard, because people will select a talk based only on the title

You know what your talk is about, and you wrote up a beautiful abstract to explain it to prospective audiences. Sadly, most conference attendees will never read that abstract. Reading an abstract is an extra click and an extra twenty seconds of reading, when people are standing in a busy corridor and they only have ten minutes in the break between sessions and they also need to find a loo and barge their way through to the pastry table and some coffee would also be welcome. Some conference apps are so clunky they actively discourage abstract-reading, even for people who already have their pastry: once you’ve paged away you can’t get back to the right point in the schedule. I love conference apps that have a tooltip hover for abstracts, but they’re in a minority.

Avoid ambiguous terms

A while ago, I went to a talk called “Take a walk on the client side.” I went because a good friend of mine was presenting it. Because I just shuffled along without thinking much about where I was going, I was expecting a talk about front-end UI development. It took me quite a while to work out what was going on in the talk, because I kept waiting for the UI to appear, and it never did. Eventually, after about twenty minutes of live demo, I worked out that in this context “client” meant “cli,” not “ui”. It’s the same word, both meanings are valid, but they’re pretty much opposites. It was an excellent talk, but it would have been even better if I’d actually known what it was about while I was watching it.

Yes, if I’d read the abstract, I probably would have understood, but I didn’t. Remember, many people won’t read an abstract. To the speaker, who was immersed in the topic, “client” was unambiguous, but for everyone else, “cli” would have carried more meaning.

Avoid cute plays on words

The other problem with “Take a walk on the client side” as a talk title is that, after going to the talk, I had the Lou Reed song stuck in my head the whole day. In fact, now that I’m writing about the talk, the song is back again. I do not want to be humming Lou Reed right now, and I blame the speaker.

A talk title’s most important job is to tell the audience what they’re going to get out of the session, and it doesn’t have many words to do it in. If all but one or two words of the title are just recycling something from J K Rowling or Tolkien or Lou Reed or whoever, it leaves very few words to explain what the talk is about. For example, in “Does my bus look big in this?” (true talk title), there’s only one word in the whole title doing real work. That’s just not enough information density.

I get why people write this type of light-hearted title. When we choose a witty riff on a well-known phrase, we are trying to communicate what people will get out of the session, but in an indirect way. We’re saying “what you’ll get out of this talk is something vague about [topic], and also a lot of entertainment, because I am Witty and Fun.” That’s a totally legitimate objective for a conference talk. However, it’s a high bar. I get frustrated if I go to a talk with a fun title, and the talk turns out not to be a-laugh-a-minute. It was a bait and switch – you promised me a joke-fest, but then gave me something dry and technical. As with everything, it is much better to under-promise and over-deliver. Choose a talk title that makes the topic clear, and then have the entertainment be a lovely surprise.

Avoid cute plays on politically-loaded words

For a while, we had a rash of “Make [whatever] great again” talk titles. I’m relieved to see that they’ve mostly passed. I’m sure when speakers chose those titles, no harm was meant, and it just seemed like an amusing play on words. Nonetheless, those words came with a lot of baggage, and this kind of title will cause problems for part of your audience. Do you really want the audience to be distracted trying to figure out if you deliberately chose a title with racist overtones or just did it by accident? No, you don’t.

Avoid flab

Many years ago I had a talk called “OSGi and Java EE: a match made in a … box?”. The information density is kind of ok(ish), because three of four words (“OSGi”, “Java EE,” and “match”) are all doing work. However, “made in a box” isn’t contributing any meaning at all. It was supposed to be adding humour, but in hindsight I think it only added confusion. If I’m honest, I got a bit stuck for a title and filled in the gap with words that seemed to fit, instead of really thinking about what point I was trying to make.

Recently, I spotted (but did not go to) a talk called “Take your shot of Vitamin!” Vitamin is not a well-known tool (I had to look it up, as would most people). “Take your shot of” isn’t doing anything to help explain what Vitamin is. Something like “How Decathlon Improve Productivity with the Vitamin Design System” is wordy, and a bit clunky, and deeply boring, but at least it would allow people to understand what the talk is about without having to read an abstract. Alternatively, “How Decathlon Improve Vitality with the Vitamin Design System” is a bit less generic, and it retains the original’s playing with the double meaning of ‘vitamin.’

Avoid cliché

There are some phrases that tend to turn up over and over again in titles. There are a 960 talks on YouTube that start with “the developer’s guide to.” There are 36 “the busy developer’s guide to.” Another common variant is “a hitchhikers guide to.” It’s kind of friendly and nice, but ultimately it’s adding words without adding meaning.

Don’t disguise product pitches

Sometimes speakers don’t just forget to include the topic of their talk, they actively omit the real topic. If your whole talk is about a product, make that clear in the title. Yes, you want to explain the benefits of the product to your audience, but don’t try and draw people in with the promise of a universal topic, and then talk about your wonder-product for forty minutes. It does not win fans.

Don’t have the same talk titles as everyone else

Now we get to the part where I contradict all of my previous advice. That’s why writing talk titles is hard! Because titles are trying to do a lot in a small number of characters, many of them pretty bland. After all, if you’re not allowed to have plays on words, or words which aren’t working hard, and you want to avoid treading in dark political undercurrents, you’re not left with much space for colour. So then conference programmes end up with columns of “Lessons learned doing [process]”, “Lessons learned: [other process]”, “[Technology]: [claimed benefit]”, “[Other technology]: [other claimed benefit]”. I have written many of those utilitarian titles myself.

It’s hard, but it is possible to do better. Devoxx recently had a talk called “Understanding Probabilistic Data Structures with 112,092 UFO Sightings”. This is a great title. There’s a clear topic and objective: how to work with probabilistic data structures. Even the ‘understanding’ part is doing work; it suggests this talk is aimed at a beginner audience, who don’t already understand probabilistic data structures. And then we move on to the colour. Including a concrete number makes the title memorable: if the it had been “Understanding Probabilistic Data Structures with UFO Sightings” it would not have been as strong. Finally, the title surfaces the fact that there will be a demo (good), and the demo domain is likely to be flying-saucer-entertaining (excellent) and pop-psychology-informative (unexpected bonus!). It would have been a shame to keep such a great demo topic buried in the abstract. But the most important thing about the title is that every single word in the title is adding something to our understanding of what to expect from the talk.

Information theory can help us understand what’s wrong with many titles. The entropy of “Lessons learned doing [process]” is slightly higher than that of “Does my bus look big in this?”, but both have a pretty low entropy. That is, if you wrote a clever compressor-for-talk-titles that had access to a pop-culture-database, they’d both compress down pretty easily. There’s a pretty obvious formula going on in both cases. On the other hand, any title including the phrase “112,092 UFO Sightings” will have a high entropy, because no other talk will have the number “112,092” in its title. (I checked on YouTube. There really are no others.)

Aim for high entropy - find a way to add personality and surprise readers while also adding meaning and without descending into cliché. If you find a way to do that, it’s a total win. Share your tips with me?

Afterthoughts: can you rename talks the second time round?

If a title doesn’t work, is it cool to change the title but keep the talk the same? I don’t know, and it’s something I puzzle over myself. Sometimes, after I’ve written a talk, I realise the title doesn’t accurately reflect what the talk turned out to be about. Or sometimes I give a talk I think is good, but audiences stay away in droves. Maybe I just chose a bad topic, but maybe the title wasn’t as compelling as it could be?

An obvious solution is to try again with a better title, but I try not to do this too often. It feels like cheating to deliver the same talk under a different title. What if program committees get confused about what they’re getting? What if someone goes to the talk thinking it’s new, and then ends up bored?

However, I do make exceptions. I wrote a talk that promised to be about testing in general, but because of time constraints, the final slides mostly focussed on contract testing. I’ve updated the title to be more specific and set expectations better. One of my favourite current talks was originally called “Writing Greener Java Applications.” It was a good talk, but every time I gave it, I was speaking to a depressingly small number of people. So I reworked the structure slightly, and called it “These Five Tricks Can Make Your Apps Greener, Cheaper, and Nicer.” It’s a total click-bait title, but I’m hoping it’s so blatant that it makes people smile. I think the title also does a better job of actually conveying what people will get out of the talk - there will be practical, actionable, advice, and the actions will be easy. It’s also a bit more informative about what’s in the talk because it introduces the idea of co-benefits (what I call “the double win”). Even if people don’t care about saving the world, there are other reasons to be green, like saving money. I talk about that a lot in the talk, so it makes sense to include it in the title.

I was prompted to write this after reading QCon’s excellent guidance for speakers about talk titles. Much of their advice is similar to what I’ve doled out myself over the years, but I re-used “Does my Bus look big in this?” from their document. Thanks, QCon!

Creating Custom QR Codes

Mon, 09 Oct 2023 00:00:00 GMT

Last week, I was at Devoxx Belgium. It was great, and I learned loads. I also have a big list of talks to go look up later. Some talks I couldn’t attend because of schedule clashes, and some I actually did attend, but I did the over-optimistic multitasking thing and realised too late that I’d missed good stuff. (Yes, I know that you too think you can multitask. No, like me, you really can’t.)

One of the talks I half-missed put up a QR code for the slides at the end, so I could catch up (hurray! thank you!). So I scanned the code, and got … this.

There was a cookie consent, for the full creepy-tracking and personalised ads level of cookie. So I noped out, and didn’t get the slides for that talk. Now I’ll never know what I missed.

I’m judgey now, but I used to use similar cookie-tastic websites to make QR codes for my slides. The reason I don’t use them anymore is that now I have my site generate its own QR codes as part of the build. (It’s kind of cool; append /qr to the URL of this page, and you’ll get a page with a QR code that points back to the parent page).

But what if, unlike me, you didn’t spend your Christmas holidays writing a big Gatsby infrastructure so that you can run custom Javascript every time you upload a talk? What if you want something in Java, not Javascript? It turns out, making QR codes is still really easy. If you can program even a little bit there’s just no need to be using web-based QR code generators. The ones you generate locally are privacy preserving, generating them is fewer clicks than fussing around in a GUI, and you can even embed your own picture into the code.

I needed to generate QR codes for the Quarkus workshops, and I was surprised how easy it was. I used JBang to make invocation easy, and used the zxing API for generation. If you’re not interested in how it works, you can jump ahead to the full source code. Download it, make it executable, install jbang if you don’t have it already, and then run

./QrCode.java <your-url> <path-to-a-tiny-image-to-embed> <output-file>

If you want an even easier execution experience, have a look at how Max Anderson uses JBang aliases to invoke the script.

What’s under the covers?

This code at the top of the class file declares the dependencies (with no build file, because of the magic of jbang), and also handles launching:

///usr/bin/env jbang "$0" "$@" ; exit $?

//DEPS com.google.zxing:core:3.4.0
//DEPS com.google.zxing:javase:3.4.0

This is the meat of the method.

 private static void writeQrCode(String text, String imagePath, String outPath, int width) throws Exception {

        Map<EncodeHintType, ErrorCorrectionLevel> hints = new HashMap<>();

        // Specify the error correction, to allow the QR code to tolerate errors, such as
        // a great big picture plunked in the middle
        hints.put(EncodeHintType.ERROR_CORRECTION, ErrorCorrectionLevel.H);

        // QR codes are square
        int height = width;

        QRCodeWriter qrCodeWriter = new QRCodeWriter();
        BitMatrix bitMatrix = qrCodeWriter.encode(text, BarcodeFormat.QR_CODE, width,
                height, hints);
        // Load QR image
        BufferedImage qrImage = MatrixToImageWriter.toBufferedImage(bitMatrix,
                new MatrixToImageConfig(
                        0xFF000000,
                        0xFFFFFFFF));

        // Initialize combined image
        BufferedImage combined = new BufferedImage(qrImage.getHeight(), qrImage.getWidth(),
                BufferedImage.TYPE_INT_ARGB);
        Graphics2D g = (Graphics2D) combined.getGraphics();

        // Write QR code to new image at position 0/0
        g.drawImage(qrImage, 0, 0, null);
        g.setComposite(AlphaComposite.getInstance(AlphaComposite.SRC_OVER, 1f));

        addOverlayImage(g, qrImage, imagePath);

        ImageIO.write(combined, "png", new File(outPath));
        System.out.println("Created QR code at " + outPath);
    }

The reason we can put pictures in QR codes is that the QR code processor treats the image as an ‘error’, and smooths it out with error correction.

        hints.put(EncodeHintType.ERROR_CORRECTION, ErrorCorrectionLevel.H);

(It turns out that ‘making the whole QR code a beautiful piece of AI-generated digital art’ can also be smoothed out as an error, but that kind of awesomeness is beyond the scope of this little utility. But go look at some of what can be produced.)

The picture you embed can be anything - a (tiny) photo, a colour drawing, or black and white vector art. Most people go for something black and white. From my own experimentation, I think that’s partly because it fits better with the rest of the QR code, visually. It also means the error correction has less to do, so your image can be bigger and still work. If your image is only black and white blocks, and the ‘correct’ content is black and white blocks, every block the QR code scanner reads has a 50% chance of being the same as what was supposed to be there anyway. If the image is colour, almost none of the overlaid blocks will be ‘correct’.

This is the code that overlays the image into the center of the QR code. If you didn’t want your own picture, you could skip this whole section.

private static BufferedImage addOverlayImage(Graphics2D g, BufferedImage qrImage,
                                                 String imagePath) throws IOException {
        // Load logo image
        BufferedImage overlay = ImageIO.read(new File(imagePath));

        // Calculate the delta height and width between QR code and the logo
        // Note that we don't do any scaling, so the sizes need to kind of
        // work together without obscuring too much logo
        int deltaHeight = qrImage.getHeight() - overlay.getHeight();
        int deltaWidth = qrImage.getWidth() - overlay.getWidth();

        int woffset = Math.round(deltaWidth / 2);
        int hoffset = Math.round(deltaHeight / 2);

        // Write the logo into the combined image at position (deltaWidth / 2) and
        // (deltaHeight / 2), so that it's centered
        g.drawImage(overlay, woffset, hoffset, null);
        return overlay;
    }

Note that this code doesn’t scale the image down, so if you pass in a huge image, you wouldn’t see any QR code under it. I did that the first time I was testing, and got confused! If the image is medium-sized you’ll see a QR code, but it won’t work properly. You can play around with finding the right size for the image. How big you can go before hitting errors will depend a bit on the exact url you’re encoding and also on the image you’re embedding. You can also play around with transparency, so that your image has a shape and not all parts are opaque. That can looks cool, and lets you go bigger.

(My code doesn’t work with SVGs, but Max Anderson has extended it to support SVGs, and also to adjust the image size in the code itself. Go have a look, it’s nice!)

Finally, the last piece of code takes care of reading command-line arguments and kicking off the image creation.

    public static void main(String[] args) throws Exception {
        if (args.length != 3) {
            System.err.println(
                    "Wrong number of arguments - should be text to encode, image path, output file" +
                            " path");
            System.exit(1);
        } else {
            String text = args[0];
            String imagePath = args[1];
            String outPath = args[2];
            int width = 640;
            writeQrCode(text, imagePath, outPath, width);
        }
    }

Save everything into a QrCode.java file, and invoke your new generator like this:

./QrCode.java http://hollycummins.com ~/Downloads/small-waving-person.png ~/Downloads/myqr.png

(This assumes you have jbang installed to handle the launch, and you’ll need to set the java file to be executable. You can also just invoke jbang explicitly with jbang QrCode.java <args>.)

Here’s what you’ll get:

Tradeoffs, Bad Science, and Polar Bears – The World of Java Optimisation

Thu, 05 Oct 2023 00:00:00 GMT

Welcome to the Java optimisation jungle. Why can’t we “just make it go faster”? It turns out, in most cases, we need to first work out “faster for whom?” and “why do we want to go faster?” and “what even is faster?” This talk introduces the basic principles of optimisation, before bouncing through the pitfalls of optimisation; why the exact same techniques which make Quarkus rocket-fast used to be a terrible idea fifteen years ago, why fast benchmarks make for slow programs, why project loom may not be the speedup you’re hoping for, and why even though it can be easy to get wrong, optimisation really really matters. Along the way we’ll talk about measuring things, bad advice, garbage collection, and climate change.

So, you want to speak at conferences

Fri, 22 Sep 2023 00:00:00 GMT

Colleagues who’ve noticed that I occasionally abandon my day-job duties and gallivant around at conferences sometimes ask me for conference tips. How can they too ditch the day job for trains, planes, and an adoring audience? I understand why they’re asking - speaking at conferences is fun, the better conferences serve dessert four times a day, and hotel breakfast buffets almost always include bacon. Speaking at conferences is also the cheapest way to attend conferences and learn.

For a long time, I was a bit unsure about what to advise colleagues. I speak at conferences, because I speak at conferences. In 2007 I submitted to JavaZone, got accepted, and since then, more speaking just … followed. Speaking at a conference created a feedback loop which made me more likely to be accepted in CFPs and more likely to be straight-out invited to conferences.

But when I thought about it a bit more, I realised my first conference talk didn’t come out of nowhere. Before I gave that talk, I’d given lunch and learns, and written articles for IBM’s developerWorks. And it turns out that when I review talks for programme committees, I look for similar kinds of experience. If I see a name I don’t know first-hand, I’ll do a search for videos of them speaking and slide decks. If I can’t find those, I’ll look for blogs. Do they have something new to say? Is this useful teaching material? Is it clearly expressed?

The speaker ladder

So although there definitely is a feedback cycle for public speaking, there are ways to get started which are accessible to all of us. These lower rungs will lead to higher levels, and, eventually, to bacon-for-breakfast and multiple-desserts.

Internal lunch and learns or tech days
Speaker coaching events (many local user groups and conferences run these to help develop new speakers)
Local user groups
Regional conferences
International conferences.
Invited conferences
Keynotes
TED talks (I guess? I haven’t got past “keynote” yet, and suspect I never will)

The pre-speaker ladder

Even before speaking, you should have something to say, and make it public. Blog, or, if you’re ok dealing with video, make those. This has two benefits; it helps you wrangle your thoughts, and it gives programme committees evidence that you know what you’re talking about. Here are some ways of sharing; as with the speaker ladder, the bottom rungs are open to everyone, and invites to the higher rungs tend to follow.

Social media threads
Self-published blogs
Articles in industry publications
Self-published books
Books with prestige publishers such as Manning and O’Reilly

What should I talk about?

One of the biggest barriers for new speakers is the idea that they don’t have anything to talk about, or that they’re not experienced enough. It’s not true! I mentioned that programme committees look for evidence you know what you’re talking about. I use the criteria “know what you’re talking about” very loosely. There are all kinds of talks, and only some involve experts:

I created [technology x], here’s what influenced the design and how to use it
I have years of experience in using [technology x] in the real world, here are the best practices
Ten ways I personally have failed at [technology x] (and how you can avoid the same failures)
I’ve just started using [technology x] and here’s all the things I had to learn for myself because the documentation assumes we’re all experts
Our industry is going about [technology x] all wrong and here’s why [technology y] is a better choice

Talks from creators and experts are great, but talks from people who’re just learning a technology are also valuable. In my experience, some experts are actually lousy teachers. They have so much knowledge that everything is obvious to them. For the rest of us, it’s useful to have things spelled out. It’s even more useful to have the pitfalls pointed out. I sometimes joke about pain-driven development, but it’s an even better model for conference speaking. Did you just have a nightmarish project? Did you just do something that was way harder than it should be? When the bad things happen, make a note, and then when you’ve recovered, come back and turn it into a conference talk. “I made this mistake so you don’t have to”, is one of my favourite genres of talk. Have you made mistakes at work? Yes? Then you’re qualified!

Where should I publish?

Before you turn all your painful projects and awful mistakes into informative conference talks, you should turn them into informative blogs. But where should you write? The best answer for this changes over time. Four years ago, I would have said Medium, but Medium’s paywall is too intrusive these days. A lot of momentum these days seems to be around newsletter platforms such as substack.

Alternatively, since you’re probably using LinkedIn already, LinkedIn articles can be a convenient place to host writing. I have a suspicion that the LinkedIn algorithm prefers links to articles over links to other platforms, so if your main audience is on LinkedIn, that can be another advantage of Articles. For me personally, I get relatively little engagement on LinkedIn no matter what platform I use to write, so I wouldn’t want to put my writing into the LinkedIn walled garden. Your mileage may vary.

If you have time, the best blogging platform is one where you own the content. Specifically, you need a copy of your writing that’s on your own machine, not on a cloud server somewhere, and the writing needs to be in a portable format. It’s happened more than once that I’ve invested far too many time carefully crafting content for a platform, only for the platform to become irrelevant or non-existent. Remember blogspot? No? Exactly. (A quick search tells me that blogspot was been bought by Google some time ago and, to everyone’s surprise, has not yet been killed. But it’s safe to say it’s not where the cool kids are.)

If you want to own your own content, that pretty much means setting up a Wordpress site, or using a statically generated site with Github Pages. I didn’t fancy paying for Wordpress hosting and I just couldn’t with all the php, so I’ve gone for GitHub Pages + Gatsby. I couldn’t find a Gatsby template that did what I needed, so I’ve written my own. Getting my custom platform set up was fun, but it did take up most of a Christmas holiday. Crafting your own platform is not the time-saving option.

Submitting to a CFP

How do I find out about conferences I should submit to?

Once you’ve got a topic and have started getting things written down, the next step is to get to an event. The great part about this is that, unlike writing platforms, you definitely do not want to self-host your own conference. Let someone else deal with the challenges of catering and venues and tickets and logistics.

But what event should you submit to? When getting started, this will probably be determined by your geography; look up the local user groups, and start attending sessions. Once you’ve got the feel for the event, reach out to the organising team and propose a talk. Most user groups will have some kind of structure for this, but it may just be an email address or named contact.

At the next level up, discovery can be harder. How do you know what the conferences are, if you’re not going to them? And how do you know when call for papers close? (In my experience, it’s usually “just before you decide to submit”.)

Every now and then, someone creates a site which gathers all of the tech CFPs in one place, but I have to say I don’t use them myself. This is partly out of laziness, and partly because I’m sceptical about accuracy. For example, confs.tech is sometimes recommended, but it’s missing both Kubecon and Devoxx, as well as Jfokus, JavaLand, JavaZone. In fact, I struggled to find a conference I’ve actually been to in the feed. The techdailycfp xitter feed has the same gap. That doesn’t mean these resources don’t have value, especially for some communities and regions, but treat them with caution.

So if there’s no one-stop website, what next? Once you have built a network of speakers, word of mouth is great for discovering good events. In the interim, follow interesting people on social media, and make a note of where they’re speaking. When you discover interesting videos, make a note of what conference they’re coming from.

How do I get accepted to a CFP?

Helen Scott has written some great tips on doing a good CFP submission. I don’t have much to add to what she wrote, apart from a plea for empathy with the programme committee. Remember that the programme committee are human, and are trying to juggle reviewing talks with their day job. Make their life easier. Don’t submit six proposals. Don’t submit proposals that are six variations on the same theme – the reviewers don’t want to be flipping back and forth between the tabs trying to figure out what the differences are. Do give them links to videos of the same talk, if you have it, or videos of other talks. If you don’t have videos, share links to writing, and if you don’t have that, explain what speaking experience you have.

(Actually, I do have one thing to add to Helen’s excellent advice - here’s some advice on crafting a good title.)

How do I get invited to invite-only conferences?

If I’m honest, I have wondered this myself many times. I’m in the lucky position that I do often get invited to conferences, and I’ve even curated invite-only tracks. Despite this, the mechanism still isn’t entirely transparent to me. There’s definitely an element of luck, along with having the right network. It’s also - as it should be - about having something interesting to say.

I often speak at GOTO. Every time they invite me it’s a pleasant surprise, and I do a little squeak of delight. But why do they invite me? At least part of the reason the GOTO team invite me is because I’ve spoken at previous GOTO events, and received good speaker ratings. So they know me, and trust that their attendees will get something out of my talks.

But if GOTO now invite me because I’ve spoken there before, how did I get invited to my first GOTO? I’d been aware of GOTO for many years, but never had an invite. Sniff. So I deployed some empathetic thinking, and some bravery. When I’ve organised speaker selection, it often happens that one or two speakers will cancel at the last minute (even before covid!). Even though speaker cancellations are inevitable, they’re horrible, and they cause panic for the organising committee. We have to scramble to find a not-terrible speaker who’s willing to step in on no notice at all. So two weeks before GOTO Copenhagen, I emailed them and said “if you happen to have a cancellation, I’d, you know, be willing to travel to Denmark even though it’s no notice at all.” I was in luck, and my timing was good. They had had a cancellation, and they hadn’t managed to organise a replacement. So I was in.

Of course, it helped that I had something to say. When I mailed GOTO, I sent them a little package with links to videos of my previous talks, and suggested a talk with an interesting-sounding title.

Geography

I’ve spoken at GOTO Berlin, Aarhus, Copenhagen, and Amsterdam. But I’ve never made it to GOTO Chicago. Similarly, I speak regularly at QCon London, but never at QCon San Francisco. What’s going on? Do I become less capable as a speaker if I cross the Atlantic? Do all Americans hate me? Well, I don’t think so. I mean, I hope not! But I suspect fewer Americans know me. And that applies to selection committees, too. I speak less in the USA, so I’m less visible, so I don’t get invited to the USA, so I’m less visible, so … it’s that same feedback loop.

Another factor is cost. Some conferences pay speaker expenses (and have higher ticket prices), and some don’t (but have lower ticket prices). If a conference pays for speakers’ travel, they’ll usually have a cap on the number of international speakers they’ll invite, and quite a small cap for the number of transatlantic speakers they can invite. Otherwise it’s just too expensive for them.

Reputation vs novelty

Familiarity clearly makes a difference to program committees. But freshness is also important. For many these events, it’s actually not about being a big-name speaker or having dozens of events under your belt. Often what they’re looking for is someone their attendees aren’t bored of! Real-life industry experience is super-valuable, even though people working at the coal faces don’t have the polished speaker skills of people for whom conferences are a day-job. GOTO and QCon do a great job of finding speakers with interesting stories or skills, who I’ve never seen before. It’s really cool.

How speakers are discovered

How do the invite-only conferences find these speakers? I’ve often wondered this. I’m sad to say that when I was doing speaker selection for one of these conferences, I really wondered it. It’s hard! I curated the Java track for QCon, and I wanted to make sure that I mixed up experienced, big-name, speakers with people who regular QCon attendees wouldn’t have seen before. So where did I look? The first thing I did was make a list of topics I was interested in; for example, I felt we’d had a lot of “how to do reactive” talks, and I wanted a speaker with some battle scars from implementing reactive in the field, who could do a deeper discussion of when reactive made sense, and when it didn’t make sense. Once I had my topic list, I started trawling for people who met the brief. Here’s where I looked:

Videos on YouTube, usually from other conferences, but sometimes just … videos
Published slide decks
Articles on publications like InfoQ and TheNewStack
Blogs
Programmes of other conferences
Social media threads

Yes, this is basically a lightly-edited combination of the speaker ladder and the pre-speaker ladder. Having content in those places increases your chance of a programme committee stumbling over your awesome stuff. For me, when I was selecting, I didn’t look on TikTok, because I’m too old. You may find that other selectors suffer from the same affliction, so know your audience when deciding where to build up eminence. At the other end of the old-school-media spectrum, if you have a recently published book, odds are conferences will reach out and invite you. On the other hand, if you have just written a book, congratulations! I don’t think you need to be reading this.

A bit like the pre-speaker ladder, even if a programme committee never finds you, having good content out there is its own reward.

Oh, and my top tip? I don’t actually have the bacon for breakfast. It’s too rich, and I need to leave room for all the desserts.

Where Can I Get Pictures for Talks?

Tue, 29 Aug 2023 00:00:00 GMT

Should your talk have images? Yes! Lots!

Before going further, stop reading this and go read the Noun Project primer on visual design for presentations. It’s got great general advice about how to make presentations look good, and also some great ‘before and after’ examples showing how to replace bullet points with images. Design matters! Presentations with good visual design aren’t just prettier; they’re also more effective.

What is the purpose of images?

Images in presentations are valuable in several different ways, and the best part is they can bring all the value all at once.

Pictures make talks better. Fact.

Here are some of the different functions of images:

Aesthetic pleasure for the audience
Reinforce your message via visual channel
Improve audience retention, because of the picture superiority effect
Hold an audience’s attention by having something change on screen
Cue for the presenter (me) about what I should be saying

The last one is important, because I try and keep the number of words on my slides low. Without words, how do I remember what I’m supposed to be saying? I’m not one of those people who can talk coherently without notes or prompts. What works for me is to build out my story with pictures, and then change slides rapidly. As a rule of thumb, I aim for two or three slides a minute, with only a couple of words per slide. The images give the talk (and me) structure.

Sourcing images

Images are great, but there’s a problem. For most of us, making new words is easy, but making new pictures is hard. Like, really hard.

The good news is, this doesn’t need to stop you having lots of pictures in your talks. No matter how un-artistic you are, you have three kinds of visual content available to you:

Icons
Drawings
Photos

Here’s how I source them, along with some of their advantages and disadvantages.

Icons

Don’t underestimate the effectiveness of a single over-sized icon or emoji on a slide. Here are some places to find them:

Your operating system’s built-in emojis (free, but limited)
The Noun Project (icons need to be paid for, or attributed in really big text; but a subscription can be worth it, because there’s a huge selection)
FontAwesome (free, if you stick to the free set)
IBM’s Carbon design icons (free, but limited selection)

The Noun Project have icons for everything, including aquariums. For this slide, I combined two oversized icons. The aquarium comes in because I was talking about a project where we used AI to count pet fish.

Note that icons are not the same as clip art. Tastes change; at the moment, icons look stylish, and clip art looks unacceptably dated. Hopefully this is obvious, but just don’t go there with anything that looks like it might possibly have once been clip art.

Drawings

Drawing it yourself is a great way of generating images for a presentation. “But I can’t draw,” I hear you wail. I can’t draw either, but practice and a series of cheat’s techniques have helped me a lot. I’ve written up my tips for getting started with drawing in “How to Draw When You Can’t Draw”.

A 2022 slide about the impact of heavy governance on IT provisioning behaviour, illustrated by me. Being able to draw something was useful, because it would have been pretty hard to find a photo of a horse running away from an open barn door. The fact that I could sneakily make the escaping horse look a bit like a Charity-Majors-approved My Little Pony was a bonus.

But honestly, drawing skill isn’t necessary. Some of the most effective presentations I’ve seen are Felienne Hermans’s. I don’t think she’d mind me saying that her drawings are not especially skilled, but they work very very well to convey her message.

Felienne Hermans at GOTO Amsterdam, June 2022

Other people’s drawings

When it comes to drawings, your own are always best, even if they’re, technically speaking, the worst. However, if you really don’t want to go there, Coco Material is a sweet little library of open source illustrations. Some of them do look a bit clip-art-y, so choose carefully. Because the drawings are simple, the Coco Material site is also a great learning resource. Study the drawings: how did they make that bottle of soya sauce so unmistakable with just a few coloured areas? How did they give that cat so much expression with just a few lines? The Coco Material license would also allow tracing, if you wanted to split the difference between drawing your own and re-using someone else’s.

Photos

I used to use a lot of photos in my presentations, but I’m now slightly cautious with them, for three reasons. The first is fashion; around ten years ago the best speakers discovered what a good idea images are, and then the rest of us started copying them. At this point in the trend cycle, photo-heavy slide decks risk looking a little dated.

The second reason is that photos have a lot of detail, and detail can distract the audience. A purer, more abstracted, visual communication can make the message clearer (Scott McCloud explains this really well, if you want to read up on it.) Photos with white backgrounds avoid some of the detail problem. Unfortunately, they’re expensive to produce, and so it’s hard to find people giving away this kinds of photo.

The final argument against photos is that they’re pretty static. I like to use lots of animations and builds in my decks. You can’t build up layers or use animations to tell a story with photos. Well, you can’t do that unless you’re a photoshop wizard, which I am not.

Those cautions aside, photos are still good! Remember, I can’t draw, so some things are just too hard to represent in a drawing. Icons lack all detail (by design), so they’re no good for representing complex things.

A slide explaining how piano stairs can encourage people to take the stairs instead of escalators. But what on earth are piano stairs? It would be almost impossible to communicate what they look like without a photo.

I usually get my photos from Unsplash. It’s also worth considering stock images, which can be pretty affordable. Be cautious with other sources, which are vulnerable to copyright predators or the opposite, image theft. Also be cautious of Unsplash images with identifiable people in them ( where a model release might be needed).

I sometimes also use Flickr, but check the license carefully. It’s best to search by license, and then search by “Commercial Use Allowed”. That should bring up images with Creative Commons Attributions licenses; but be careful to attribute correctly. I usually avoid Creative Commons Share Alike for my presentations, since I don’t necessarily want to release them under a creative commons license.

Two sources of public domain images I’ve recently discovered are public.work (a beautiful aggregator) The Library of Congress free-to-use archive. The images tend to have a vintage feel (because they’re older and thus out of copyright), but there are also some new ones. There’s a range of bridges, public service posters, landmarks, and all sorts of other potentially useful images.

For example, these are two of my favourites, and I can imagine using either in a talk:

A woman operating a hand drill at Vultee-Nashville, 1943. With the red head scarf, it gives the impression of a real-life Rosie the riveter. I love the contrast between the impeccable manicure and heavy industrial context.

Many of the images in the Library of Congress archive would give a slide a dignified or inspiring mood, but not all. There’s a whole category for cats:

A cat, improbably wearing a winged helmet and breastplate armor in the role of the valkyrie Brünnhilde from the opera Der Ring des Niebelungen, 1936. I wonder if the cat also sang?

Visual consistency

Aim for consistent visual style, or a limited mix. Having icons on one slide and drawings on another can look sloppy and unpolished. One of the first things I think about when starting a talk is whether a topic is best suited to icons, or drawings, or photos. That ensures I don’t spend time making drawings which I later have to rip out and replace with photos, or the other way around.

AI Image Generation

What happens if you’re trying to stick to a style, but you can’t find what you need? I’ve started experimenting with tools such as DALL-E or Stable Diffusion for generating images. Sometimes the results are astonishingly great, and sometimes they’re entertainingly awful.

I think we can all agree this is not the Duke, the Java mascot, with grey hair and a beard:

DALL-E output for “A version of the Java Duke Mascot, but with grey hair and a beard”. Yes, the text appears to say JAJAAVVE.

On the other hand, if you need a photo of a duck wearing a fedora, you won’t do better than this unless you’re a skilled photographer with access to well-trained ducks:

DALL-E output for “photo of a duck wearing a red fedora white background”

Although image generation is cool, be aware that generating images is energy intensive, and most generators were trained by scraping copyrighted artists’ work without compensation. That’s less cool.

Hybrid styles

Sometimes it just doesn’t work to stick to only-photos or only-drawings. It can be quite effective to layer line drawings or icons over photos. It can also look fresher and more modern than plain photos. For an example, take a look at the first image in this Noun Project article, or this slide, where the fire icon reinforces the meaning of the fire photo:

A slide from a 2019 talk. The icon and image reinforce each other.

On the other hand, in this slide the photo and icon are complementary. The photo contributes one piece of information ( what a Bramble Cay melomys looks like, because people are unlikely to know), and the icon adds extra context (uh oh, it’s extinct).

Another slide from the same talk. The icon and image each tell a different part of the story.

Image selection

We’ve talked a lot about how stuff looks. But what about the actual content of the pictures?

Figuring out what image goes with what concept is hard! Sometimes I have a good idea when drafting out the talk, but more often I draw a blank (do you see what I did there?!). When I’m stuck, I just leave a ‘TODO graphic here’ and then alternate runs and showers until inspiration hits. When choosing an image, think about the purposes of the image; does it need to convey extra meaning, or is it just there to entertain the audience?

Some topic-and-image associations have become clichéed. Instead of choosing a predictable image, can you find something unexpected which will add an extra layer of meaning, or make people laugh with an incongruous association? For example, if the topic is “working at home”, the obvious image might be a desk with a computer on it, but maybe you could show an unmade bed, or a pair of comically fluffy slippers. When I talk about under-utilized servers, I don’t show pictures of data centres or load meters; instead, I take the opportunity to draw cute zombies.

A slide from 2020, with a hand-drawn zombie. Notice that the visual style combines the shape of icons and a rough hand-drawn finish. I traced icons to get some of the proportions right.

Here’s a later zombie illustration, conveying the same idea in a different style. Another zombie slide, from 2023. The zombie looks a little bit like a grinch, which wasn’t my intention, but at least it makes people smile.

Have I mentioned already that I’m not an expert on this? I attempt to draw a zombie and end up with a grinch. I’ve shared the patterns that work for me, but styles evolve, so I expect in a few years time I’d have a different set of recommendations. I aim to keep experimenting, and when I see a visually effective talk, I try and figure out what makes it work. There’s no right answer, but we can always learn from others. As a final reference, Understanding Comics by Scott McCloud is notionally about comic books, but it’s actually a great deep dive into drawing and visual communication. I loved it.

Pact Contract Testing for Quarkus

Wed, 26 Jul 2023 00:00:00 GMT

Every time you change one microservice, others break. But you had unit tests! What’s going on? Unfortunately, unit tests aren’t enough to give system-level c…

How to avoid common pitfalls with modern microservices testing

Tue, 04 Jul 2023 00:00:00 GMT

Have you ever wondered why your microservices break in production after all the tests have passed during CI/CD? Have you ever wondered if you’re testing too much or too little? If you want to learn about common pitfalls and how to catch them at build time rather than at runtime, this session is for you!

In this session we’ll share some common battle scars from our experiences in the field. Additionally, we will introduce the testing pyramid and explain why it is important in helping to continuously deploy microservices independently without fear. We will demonstrate, through a test-driven development approach, how the combination of Pact contract testing combined with Quarkus developer joy can help prevent your microservices from breaking in production.

Quarkus and Contract Tests with Holly Cummins

Thu, 08 Jun 2023 00:00:00 GMT

Our guest today is Holly Cummins, a Java Champion, developer, author, and a Senior Principal Software Engineer at Red Hat working on Quarkus. Today we’ll tal…

Redis, Quarkus, OpenShift From Cache to Modern Apps at Scale

Tue, 06 Jun 2023 00:00:00 GMT

Redis, Kubernetes OpenShift, and the Quarkus Java stack is a powerful combo for deploying highly available, clustered databases at scale.

Quarkus 3 Technical Spotlight: Pact contract testing with Holly Cummins

Tue, 23 May 2023 00:00:00 GMT

Using Pact and Quarkus to Tame Microservices Testing

Mon, 15 May 2023 00:00:00 GMT

In a microservices architecture, making sure each microservices works is (relatively) easy. The microservices are usually small, and easy to test. But how do you make sure the microservices work together? How do you know if the system as a whole works?

Holly Cummins at Devoxx UK: How Would the Business Benefit from Your Greener Java Application?

Sat, 13 May 2023 00:00:00 GMT

At her Devoxx UK presentation, Holly Cummins, senior software principal engineer at Redhat, presented approaches that could make Java applications more cost and energy efficient. Moreover, the busines

How to avoid common pitfalls with modern microservices testing

Thu, 11 May 2023 00:00:00 GMT

Wicked Good Development Episode 32: Java Queens at Devnexus 2023

Fri, 28 Apr 2023 00:00:00 GMT

Kadi Grigg and Theresa Mammarella sit down for a conversation with Erin Schnabel, Emily Jiang, Mary Gygleski, and Holly Cummins at Devnexus 2023.

Resources for Writing Quarkus Extensions

Thu, 20 Apr 2023 00:00:00 GMT

I’m sometimes asked for a list of resources for writing Quarkus extensions. It’s a topic I’ve been meaning to write more about, but I haven’t yet, because as Emmanuel Bernard says, the todo list is infinite. Every time someone asks me, I re-assemble my list of favourite links. To save redoing that step each time, I’m writing the list down in a permanent place.

The Quarkus Team have two guides about writing extensions, a simple one, and a deeper one.

The deeper guide is now linked from the top menu bar of quarkus.io as “Create Extensions” to make it easier to find.

As of autumn 2024, there are also now two companion guides:

I wrote them, so I think they’re useful. :)

I sometimes think we could also use a “my second extension” guide, but we haven’t written one yet. (Remember, the todo list is infinite.) However, there are some great community blogs/instructions that fill in that gap:

A walkthrough of writing an extension to log to Amazon cloud watch, which is an excellent general tutorial about writing an extension. This is the tutorial I followed when writing my first extension.
A thorough walk through of writing an extension, including a dev service. This is relatively recent, and it’s great.
If you prefer video, Sebastian Daschner has a nice walkthrough of creating a simple extension which integrates to a LED light to show application health
How extensions can help with byte-code rewriting and other similar use cases:
- Stripping out methods which have a problematic annotation
- Resolving an annotation conflict by transforming breaking annotations into happy ones
My and Georgios Andrianakis’s talk about writing extensions with an introduction to writing extensions. The demo is using minecraft for logging. I’m hoping to keep the source code for that extension up to date as a bit of a reference for some extension scenarios. Apart from the chicken-proliferation-jokes, this is my favourite bit of that talk, and it’s not in other resources, so I’ve included it here as an animation.

One of the most important resources is the list of all build items, the guide which lists all build items. The build items are a bit like the SPI for extensions. Almost everything that an extension might want to do can be handled by an existing build item or chain of build items. The trick is knowing which one! It’s a good idea to browse the list of build items to see if there’s one which meets your needs when you get started. I also re-review the list every now and then to make sure there’s nothing I’ve forgotten or missed.

How to avoid common pitfalls with modern microservices testing

Wed, 05 Apr 2023 00:00:00 GMT

Happy Path podcast

Wed, 05 Apr 2023 00:00:00 GMT

Duke's Corner podcast

Mon, 03 Apr 2023 00:00:00 GMT

In this conversation Oracle’s Jim Grisanzio talks with Holly Cummins about her passion for being “in the engine room of Java” in that she’s trying to build tools and frameworks to reduce complexity for developers and increasing their productivity. The conversation also touches on the Java community, Open Source, and various software development practices. Holly is a Sr. Principle Software Engineer at Red Hat in the UK. You can find her on Twitter @holly_cummins. You can find Jim at @jimgris on Twitter.

Why Cloud Zombies Are Destroying the Planet and How You Can Stop Them

Fri, 31 Mar 2023 00:00:00 GMT

At QCon London, Holly Cummins, Quarkus Senior Principal Software Engineer at RedHat, talked about how utilization and elasticity relate to sustainability.

Expert Talk: War Stories from Moving to the Cloud • Holly Cummins & Lorna Jane Mitchell • GOTO 2022

Tue, 14 Mar 2023 00:00:00 GMT

This interview was recorded for GOTO Unscripted at GOTO Copenhagen. #GOTOcon #GOTOunscripted #GOTOcphhttp://gotopia.techRead the full transcription of this i…

Why Your Desk is the Worst Place to Work, and Other Life Lessons from a Lazy Developer

Thu, 09 Mar 2023 00:00:00 GMT

Quarkus Insights #117: Contract Testing with Pact and Quarkus

Mon, 13 Feb 2023 00:00:00 GMT

Holly Cummins discusses how contract tests combine the lightness of unit tests with the confidence of integration tests, and should be part of your development toolkit.

Modern Java runtimes: No Regrets energy efficiency with Quarkus and GraalVM - A Redmonk Conversation

Thu, 15 Dec 2022 00:00:00 GMT

In this video we talk to Red Hat’s Holly Cummins about modern Java runtimes and how they can make your software and systems more efficient. Small, lightweigh…

Sustainable Software and Systems: Lightswitch Ops for the Triple Win - A RedMonk Conversation

Thu, 15 Dec 2022 00:00:00 GMT

In this video I talk to Red Hat’s Holly Cummins about sustainability, the imperatives for greener software, and how to achieve it with modern Java architectu…

Sustainable software and systems, efficiency and the triple win. Policy, progress, and modern Java runtimes like Quarkus.

Thu, 15 Dec 2022 00:00:00 GMT

The imperative for more sustainable and power efficient applications and systems is stronger than ever. Not only is the green agenda becoming more pressing, but recent energy price spikes mean that…

Tradeoffs, Bad Science, and Polar Bears – The World of Java Optimisation

Wed, 30 Nov 2022 00:00:00 GMT

Expert Talk: Cloud Chaos & How Contract Tests Can Help • Holly Cummins & Kevlin Henney • GOTO 2022

Fri, 25 Nov 2022 00:00:00 GMT

This interview was recorded at GOTO Amsterdam 2022 for GOTO Unscripted. #GOTOcon #GOTOunscripted #GOTOams

How to write greener Java applications

Fri, 11 Nov 2022 00:00:00 GMT

There is a need to reduce the impact of the software industry on the environment. Read to discover more about how Quarkus reduces carbon emissions.

The Sad Case of the Tech T-Shirt (Fashion and Programming, part ii)

Tue, 01 Nov 2022 00:00:00 GMT

A year or two ago, I wrote about the connection between fashion and programming. For reasons I don’t totally understand, it’s my second most popular blog post, after my leaving IBM announcement. Apparently, people do searches for ‘fashion coding’ and ‘coding in fashion’. There’s not many results for that kind of search, so my site is the top result. A coding school has even linked to the article.

When I was writing that article, I almost deleted it several times, because it was so personal, and frivolous, and resentful. This one has almost been deleted too, for the same reasons.

I started thinking about fashion and developers again because I saw Purity Birir’s tweet. It said in 240 characters what I took a whole blog to try to say:

This isn’t just about clothes; RedMonk have written about the Developer Aesthetic without a single mention of hoodies. It’s about how communities create their own visual codes, and how that the design choices we make as individuals communicate that we belong in a group. Or, if we’re unlucky, the choices give an impression we don’t belong. (Light mode user here, hi!)

But back to clothes.

Purity is right that as a woman in tech, there is definitely a ‘wrong’ outfit. (Too short, too shiny, too frilly, too female, basically.) Women who wear the wrong outfit will be Told.

… but it’s worse than that. There definitely is a wrong outfit, but there isn’t even a ‘right’ outfit. No matter what we wear, someone might get distracted.

I’ve been thinking about this for a while, but I had a reminder this week. Eric Deandrea and I spoke about contract testing at Devoxx Belgium a few weeks ago. This week, Eric spotted this comment on our talk video:

So far, there have been no comments about Eric’s outfit, even though we were wearing almost identical clothes.

Photo by Devoxx Belgium

The comment was moderated out as soon as Eric reported it, but heads are harder to moderate. I’m still sad, and mad, because I try very hard to choose innocuous outfits that let me blend in with the tech herd. It’s disappointing to be reminded how totally I am not-succeeding at blending in.

Ok, if I’m totally honest, I could try a bit harder to blend my shoes in. Here’s a still of the “high heels” I’m wearing in the video:

I usually wear block-heeled boots, unlike most of my tech colleagues. On the other hand, my experiments with flatter shoes have not been particularly successful. I clomped around in Doc Martens for half of Devoxx, in an attempt to blend in. I was uncomfortably short and the heavy Docs shredded my feet so thoroughly I still have blisters several weeks later.

I do agree with the commentator about one thing: I’m not sure the Quarkus t-shirt I’m wearing is my best look.

I have a conflicted relationship with tech-product t-shirts. Branded t-shirts are a standard speaker uniform at conferences. I wear them because my male colleagues do, and because I want to fit in, and because I’m proud of my product. But branded t-shirts look very different on women than they do on-men.

The first issue is even finding one that’s an appropriate fit. As Kathy Sierra pointed out, many tech t-shirts are only printed for men. A small version of a straight-cut men’s fit t-shirt will usually be too small in the neck (aurgh, strangled!), too small in the hips, and too small in the bust. A medium version of a men’s t-shirt will be a tent.

Even when branded t-shirts physically fit, they can be uncomfortable. GeekFeminism did a great analysis of the hazards of having a bust. T-shirts with a picture over the chest area, by definition, draw attention to the wearer’s chest. That’s not a problem for men, but it is for women. Because women tend to have bumps on their chest, logos sometimes deform in weird ways, and really draw attention to the breasts. That’s the exact opposite of the “blending in” that I’m hoping for.

There’s a more subtle problem, too. Tech t-shirts just aren’t my natural style. I find them hard to make into an outfit. That matters, because “borrowed someone else’s t-shirt for the day” is not a look which says “expert on the subject”. Tech t-shirts look ridiculous with most of the trousers I own, apart from one or two pairs of jeans.

Why can’t I just wear what I want? Well, it means I’m not representing. And if I wear the clothes I feel most comfortable in, I get told I look corporate. This isn’t a hypothetical; it happened a few weeks ago while I was in a conference hotel wearing my comfy home clothes, on an ‘off day’ between events. No malice was intended, I just … did look kind of corporate.

What’s the solution? I don’t really know. We need to focus on what women speakers say, not their shoes or t-shirts. We need to embrace the #ilooklikeadeveloper hashtag. We need to let our clothes out of the closet. And we need to normalise diversity in tech - not just of people, but of outfits.

Innovation Anti-patterns

Fri, 28 Oct 2022 00:00:00 GMT

In a world where there’s always too much to do and too little time, how can organisations make time to innovate? (Spoiler: siloes probably aren’t the answer.) In this article we’ll explore practical innovation strategy, exploring how to design an MVP (good), and how to get something for nothing (great!). But before we do, let’s talk about George Washington Carver.

Tradeoffs, Bad Science, and Polar Bears – The World of Java Optimisation

Fri, 14 Oct 2022 00:00:00 GMT

How to avoid common pitfalls with modern microservices testing

Thu, 13 Oct 2022 00:00:00 GMT

Expert Talk: Cloud Chaos & How Contract Tests Can Help • Holly Cummins & Kevlin Henney • GOTO 2022

Tue, 04 Oct 2022 00:00:00 GMT

This interview was recorded at GOTO Amsterdam 2022 for GOTO Unscripted. #GOTOcon #GOTOunscripted #GOTOams

https://gotopia.tech

Read the full transcription of t…

Refactor This Podcast - vFunction

Thu, 15 Sep 2022 00:00:00 GMT

Refactor This by vFunction brings the world of app modernization into the spotlight through friendly conversations with industry luminaries.

Uh, Does This Thing Actually Work? Modern Microservices Testing

Wed, 31 Aug 2022 00:00:00 GMT

To realise the promise of microservices, individual services must be independently deployable, and they should be deployed often. But that’s easier said than done. Releases take confidence. How does a business achieve enough confidence to release often without getting bogged down in manual QA bottlenecks? Is automating tests too expensive? How do services know if they’ve made a breaking change? If the individual microservices all work, does that mean the system works? This talk discuss microservices testing strategies and explore some of the modern tools which make testing easier and better. And there’s a demo: Holly will show Pact contract testing, test containers, and Quarkus’s continuous testing and dev services support.

Tradeoffs, Bad Science, and Polar Bears – The World of Java Optimisation

Tue, 23 Aug 2022 00:00:00 GMT

[Charity conference for Ukraine]

Hackerstations: clean and minimalistic setup in London, UK

Thu, 11 Aug 2022 00:00:00 GMT

Quarkus Insights #98: Using Minecraft as an Observability client

Mon, 08 Aug 2022 00:00:00 GMT

In Quarkus Insights Ep.98 Holly Cummins joins the team to discuss her Quarkus observability demo that uses Minecraft as a client.

Around IT in 256 seconds #81: Quarkus: supersonic, subatomic Java

Fri, 05 Aug 2022 00:00:00 GMT

Author: Holly Cummins

Amigas: programar software no es más difícil que aprender idiomas

Sat, 23 Jul 2022 00:00:00 GMT

Qué hace para ganarse la vida? Explíquenoslo a mi abuelo y a mí. Soy ingeniera de programación informática en Quarkus. Habrá oído hablar del lenguaje Java… Hasta

Why You Can't Buy Cloud Native

Tue, 19 Jul 2022 00:00:00 GMT

The virtual shelves are filled to bursting with cloud native technologies; containers, runtimes, CI pipelines, observability tools. The more an organisation buys, the more cloud native it is, right? And the more network communication an app involves, the more cloud native it is, right? Well, not so much. It turns out cloud native is about how you deliver software, not the tools you use. This talk explains these often-overlooked elements for cloud native success:

Figuring out what problem we’re trying to solve
Releasing
Testing, but not the way you’re used to

Stackd 59: Holly Cummins, Quarkus, IBM, and the Garage

Mon, 18 Jul 2022 00:00:00 GMT

Kito, Danno, and Josh welcome special guest Holly Cummins, Senior Principal Software Engineer on the Quarkus team at Red Hat and fellow Java Champion, to tal…

Getting Into and Testing Microservices

Tue, 12 Jul 2022 00:00:00 GMT

Senior Principal Software Engineer Holly Cummins teaches us about microservice architecture and how to test it.

How to draw when you can’t draw, part ii: no really, how to draw

Fri, 17 Jun 2022 00:00:00 GMT

In part i, I explained why I started drawing despite not having much artistic talent. Practice has helped me move from ‘totally inept’ to ‘mostly inept’. Here are some of the other techniques I’m using to improve.

2021, self portrait. Learning from failure is important. Laughing at failure also helps.

Observing

Carol Dweck observed that part of a growth mindset is seeking out people who do the thing you’re trying to do better than you can do, so you can learn by example. (In contrast, people with a fixed mindset tried to cheer themselves up by looking for people who were even worse.) Some of the improvement in my drawing has come from just doing the thing, over and over again. But I’m also trying to be a bit Sherlock-Holmesy and observe, rather than just seeing. When I see a drawing I like, I study it. What about it do I like? What techniques did they use? Could I ~~steal~~ use those techniques?

Sometimes, the answer is ‘no’. I’m limited. Anything detailed or life-like or complex is beyond me. But even drawings with only a few simple lines can be powerful. For example, Charlie Mackesy’s pictures only have a few splotches of colour and squiggles but they’re so evocative. (I haven’t embedded any of his pictures to respect his copyright, but do go have a browse of his site.)

Education

I assumed drawing was an innate talent, but it can be taught. Here are some of the resources I’ve been learning from:

My colleague Jocelyne Dittmer wrote a great guide to sketch-noting for non-drawers. Because the visual thinking skills are similar, it’s also useful for slide illustration.
‘How to draw’ videos for kids are about the right level for me. I started to watch them because I had young children, but now I watch them on my own. My favourite is Art For Kids hub.

One of the things on my todo list is to explore other drawing tutorials. Red Hat do regular Days of Learning, so perhaps I’ll take my next one for drawing instruction. I might even be confident enough to work through lessons aimed at adults.

Experiment

Experimentation is a key part of continuous improvement. Over the years, I’ve tried many different styles of drawing – with shading, flat, monochrome, white backround, black background, bold lines, fine lines …

For my Kubecon keynote I tried icon-shaped people instead of stick figures:

2020, a zombie with an icon-like body shape

I used loose sketching instead of thick lines:

2020, a melting planet

Some of my visual experiments haven’t even involved drawing – for example, I like how slides come out if I just use some super-sized icons from the noun project.

2019, a huge icon in place of hand-drawing

Even after all my practice, there are some things I just can’t draw. Rather than going all the way back to just-photos, I’ve had good experiences with a hybrid style. I think it’s more expressive than either photos of drawings alone.

2020, annotation animations on top of a photo

The tension between the two media gives the slides an extra energy, I think.

2020, a hybrid illustration

Hybrid works well with icons and photos, too.

Not every experiment is a success. I had to abandon a new style of stick figures halfway through drawing a presentation because they were all turning out improbably phallic. “NSFW presentation through incompetence” is a milestone I hope to avoid.

The mechanics

People sometimes ask me about my drawing process, so here are some details. I started out drawing on paper, with sharpies, and then using a scanner to digitise them. It worked, but it was labour-intensive, and I could only do it at home. No last minute revisions to the slides for me. Not that I would ever leave a presentation to the last minute. Obviously.

An example drawing done with Sharpie on paper, 2015

Eventually phone cameras got good enough I could use them instead of a scanner, so that made the process a bit more portable, but it was still slow. I saw Marlena Compton do some amazing sketching using the Paper stylus and an iPad app, so I switched to digital. (Marlena can even give a talks by live-drawing them, which is something I’ve never been brave enough to attempt.) Going digital meant I could trace from photos, and also experiment with different brushes and ‘pens’ beyond just sharpies.

An drawing using the Paper app and stylus, 2016

However, the first app I used was more targeted for note-taking, so my drawings were disappointingly pixelated on a big screen.

Look at all those pixels on the diagonal lines, yuck.

When the iPad Pro and Pencil came out, I switched and haven’t looked back. Even though I only use a fraction of the power of my drawing app, I can use layers to undo errors or build up animations, I can shade, I can smudge … I use Tayasui Sketches Pro. I like it, but I haven’t evaluated it against other apps, so it’s not exactly an expert recommendation.

As an example of what the drawing app can do, in this drawing I used stippling to give the feathers some texture.

2021, a duck.

Here I used blending to try and convey depth.

2021, a hole.

Tracing

How can I draw complex shapes? I cheat. I hate to say ‘fake it ‘til you make it’, but this is basically that. I trace. If I’m drawing something beyond my skill (a horse, a cat, something with perspective … well, if I’m honest, almost anything), I find a suitable photo to use as a base. (This is something digital drawing tools make very easy.) If a single photo isn’t enough, I’ll patch a few photos together. I’ve also used setpose for getting people-proportions right. It’s a digital version of the poseable mannequins artists use.

A traced sheep with a freehand wolf, 2022.

I feel a bit sheepish about tracing, but who among us hasn’t copied and pasted from stack overflow? And learned something in the process? Historically, artists were trained by copying old masters. Many still learn by copying, a practice Austin Kleon defends in Steal Like An Artist and blog posts.

I’m not sure how many artists trace, but street artist Shepard Fairey famously traced a photo for his ‘Hope’ Obama poster. (Fairey was then famously sued by AP, so even when tracing, copyright matters.)

I’ve also been told that working graphic designers trace all the time. They’re trying to do a job, so they use the tools at hand. I think the same holds for me; I’m an engineer, and the reason I give presentations is to communicate complex technical concepts. If tracing helps me do that more easily, I’m happy.

Postscript: Other tools

If you’re not ready to dive into a full-on tablet-and-drawing-app, there are tools available which can create hand-drawn-style graphics. They’re opinionated, so the stylistic options are limited, but the results are great, they’re quick, and they’re easy. I was impressed by Excalidraw, which was a recommendation from my colleagues John O’Hara, and, indirectly, Gunnar Morling. It’s a free web page, so the barrier to entry is basically zero. A handy thing about Excalidraw is that there are open libraries of icons, with a lot of tech icons available to drop in to diagrams (including Quarkus, naturally). Perhaps even nicer, if you paste tabulated data onto the canvas, it will auto-generate cute hand-drawn-style charts.

How to draw when you can’t draw, part i: a growth mindset story

Thu, 16 Jun 2022 00:00:00 GMT

I’ve never been particularly artistic. As a kid, like all young children, I liked drawing. At some point, probably once I started having art class in school, it became obvious to me I wasn’t very good at it. My more talented peers were producing beautiful pictures, and clumsy me was drawing lumpy figures that only barely looked human. Luckily, I knew the solution. I gave up.

An early effort, drawn when I was six and a half. The thing in the air is an (inexplicably large) bee.

I stopped even trying, because if I wasn’t trying, I couldn’t fail. I never drew in my spare time, and as soon as art classes weren’t compulsory, I dropped them. Instead, I focussed on the things I knew I was good at (science, math, English … definitely nothing that required hand-eye coordination).

I didn’t even miss the drawing I wasn’t doing until a few years ago. I started doing more conference speaking, and I needed lots of pictures. It’s well known that visuals make talks more enjoyable for the audience. Well-chosen visuals can actually make talks more effective, too – reinforcing a message across both visual and verbal channels makes it sink in more. People remember visual messages better than they do verbal-only ones, a phenomenon known as the picture superiority effect. Part of the reason for this is that images seem to be a ‘native format’ for our brain’s memory, just like words (dual coding theory).

2021. Talks work better with graphics.

Like many other presenters, I used photographs in my presentations. But the right photograph can be hard to find, and copyright predators are an increasing concern. You’re limited in what you can express with existing photos. I tried taking my own photographs, but staging photographs is hard work. My breaking point came when I spent hours and hours making batches of coloured fresh spaghetti and arranging it in different configurations to illustrate classloaders, jars, coupling, testing and services for a talk on software modularity. I even invested in a light tent. The photographs were … not great. (It turns out that even though pointing a camera and pushing a button is easy, photography is a skill.)

Homemade spaghetti in a homemade photograph, 2012

I was also conscious that when I used photos, even if the photos were good photos, my presentations looked exactly like everyone else’s.

So ten years ago, I tried drawing my own slides. I was drawing at about the level I had when I gave up on art, in primary school. Nonetheless, I was impressed at how flexible sketches were for communicating technical ideas:

Even with my clumsy drawings, I was also able to express emotion. And drawings lent themselves to animations which built up a story in a way photos didn’t:

Here’s what I’ve learned over seven years of drawing:

Difference is good

If a presentation looks exactly the same as six other presentations a conference-goer has seen that day, it’s unlikely to be memorable. If it looks exactly the same as the presentations they were seeing five years ago, that’s even worse. Just yesterday, after a talk, someone said to me “I enjoy that your slides don’t look like the others.” There’s a lot to be said for novelty.

(I should note that ‘different’ is a relative term. Other speakers, such as Kasia Mrowca and Felienne Hermans, also do great hand-drawn slides, and I’ve learned from them.)

Competence is over-rated

Most of us naturally avoid doing things we’re bad at, especially in public. We don’t want to be inadequate, and we don’t want to be caught out as inadequate. But going outside our comfort zone can be good for us, and good for those around us. For example, I find the most helpful conference talks are from people learning a topic, because they dig into the non-obvious points and gotchas. Similarly, new joiners to a team make a huge contribution because they spot gaps and bring fresh ideas.

My artistic mediocrity may bring a similar ‘win’. When I give a talk, people enjoy my drawings, but no one in the audience is going to be dazzled by the artistic majesty of my creations. My illustrations serve my technical content, because they’re so simple they can’t do anything else. If, on the other hand, Leonardo Da Vinci was trying to explain microservices anti-patterns, people might be distracted by the Mona Lisa on the slide in the background.

2021. Abstract and low-skill, but still effective at communication.

That doesn’t mean incompetence doesn’t have its downsides; occasionally, people just end up confused. I was so pleased with a drawing I did for a Javaland keynote of someone cuddling a cloud that I put it in my email signature. I hastily took it back out after the third person asked me why I was sharing a picture of someone holding a large loaf of bread.

Extract of my email signature, 2018. Love cloud, or love bread? You decide.

It’s uncomfortable doing things badly, and most of us try to avoid it. But sometimes doing something badly is better than not doing it at all. And doing it badly is how we learn, which is how we get better.

Practice

In the IBM Garage, we used to tell a story about a university pottery class which was divided into two groups. Half were asked to produce a single pot, and graded on the quality of that pot. The other were graded by pottery weight; the more pots they produced, the higher the grade. It’s obvious that this is a terrible way of grading work; what’s less obvious is that the by-weight group ended up producing work of a higher quality!

How can incentivising quantity possibly create quality? And is the story even true? Austin Kleon looked into the pottery-grading facts. The story turns out to be mostly-true, but the original medium was photography. Rewarding quantity created the conditions necessary for students to get lots of practice, which gave better end results. The other group focussed on trying to get the one-perfect-shot learned less and so didn’t actually get the perfect shot. For a lived example, Amy Iskioff Newell wrote a wonderful blog exploring how taking 33,000 selfies in the pandemic grew her photography skills. No matter what the form, the underlying – verifiable – lesson is that the more we do something, the better we get.

2018, CV-driven development as a driver for container adoption. The image on the left was my first attempt, but I tried again and got the image on the right. I think it’s much better.

“Practice improves us” is pretty obvious, but we’re not very good at believing it. Many of us tend to think that our aptitudes are inborn and our blind spots are innate. This is no doubt partly true, but not entirely true. How true is ‘partly true’? For practical purposes, it doesn’t actually matter.

What matters is how elastic we perceive our abilities to be. And that matters a lot, like “near-instantaneously changing test scores matters”. A one-hour lecture on how the brain is like a muscle and our abilities aren’t fixed significantly improves teen’s academic results over a whole school term. How significant? One model estimates the effect is comparable to the difference between an average and good teacher for a whole year.

I often need to illustrate elasticity. This is my best attempt so far, but it’s … not great. I’m sure I can do better.

Just as the right statement can improve results, the wrong statement can worsen results. Telling a child something about the abilities of their group hinders their performance on challenging tasks. For example, telling a girl “boys do well at this puzzle” just before asking her to do the puzzle makes her do badly at the puzzle. That’s not totally surprising, since it’s well-known that confidence affects performance. What’s more surprising is that if you repeat the experiment and tell a boy “boys do well at this puzzle,” he’ll also do worse. The way to undo the effect is to provide an explanation, and say something like “boys do well at this task because they try hard.”

Linda Rising has several other examples in her ‘Agile Brain’ lecture, which is where I first heard about growth mindsets¹. Carol Dweck, who pioneered growth mindset research, suggests thinking in terms of “yet”. “I’m not good at drawing … yet.”

If we think our abilities are fixed, when something gets hard, we decide we must not be good at it. We stop trying. Instead of seeing failure as a useful part of the learning process, we treat it as a terminal indictment of our abilities and potential. In fact, if we think our abilities are innate, we don’t try as much even when we seem to be doing well. This hurts us, because effort and practice are (duh) important for improvement. As Amy Iskioff Newell says,

“You can get better at anything if you practice a lot, even if you don’t have much of a plan for how to get better, even if you don’t set any goals, take any classes, read a single book or watch a single video on even the most basic topics.”

Looking at the difference between my 2015 drawings and what I’m doing now, I can see how much practice has helped.

Burger, 2021. I have more confidence with reflection and shading than in my earlier pictures.

Have I got a perfect growth mindset? Of course not. While writing this blog, I happened to see some of @MalwareUnicorn’s drawings on twitter, and I got totally discouraged. How can I have the nerve to draw in public, and then brazenly write about my drawings as if I’m an expert, when there are people who draw so so so much better than me? In fact, should I even be allowed to draw at all? Even Amanda’s casual doodles are better than my best ‘after’ picture, and I have to cheat.

I will never ever draw that well, growth mindset or not, practice or not. But I’ll keep trying. Another finding of Carol Dweck’s is that, after a disappointing result, students with a fixed mindset tended to seek out work from others who’d done worse, so they could feel better about themselves. Those with a growth mindset wanted to look at what people who scored higher produced, so they could get better. So, yeah. I’m never going to be a @MalwareUnicorn. But I’ll keep scribbling, and see where I get to.

In part ii, I discuss some of the mechanics behind my drawing, including the (sssh) cheating.

Like a lot of widely-reported psychological research, ‘growth mindset’ has a bit of replication crisis. I found this essay nuanced and informative: Is Growth Mindset Real?. The conclusion is that there isn’t much correlation between self-reported growth mindset and academic results. However, teaching people they can try hard and grow their abilities does lead to improved results. So it’s not great as a predictor, but it’s excellent as an intervention.↩

Why you're missing out if you're not making Quarkus extensions

Tue, 07 Jun 2022 00:00:00 GMT

Do I need an extension? And how do I write it?

Extensions are a great way of enabling your favourite library to fully take advantage of Quarkus’s capabilities. They can also be used to improve your team’s everyday workflow.

In this talk Georgios and Holly will talk through how extensions can be useful, before live-coding a simple extension. ( What it does is a surprise!)

Why You Can't Buy Cloud Native

Tue, 07 Jun 2022 00:00:00 GMT

Figuring out what problem we’re trying to solve
Releasing
Testing, but not the way you’re used to

Innovation Anti-patterns: Bonkers Beans and Magic Numbers

Fri, 03 Jun 2022 00:00:00 GMT

The innovation fizzle, the fauxnnovation, the maximal viable product. Holly’s seen them all. In a world where there’s always too much to do and too little time, how can organisations make time to innovate? (Spoiler: siloes probably aren’t the answer.) A presentation on practical innovation strategy, exploring how to design an MVP, how to get something for nothing, and psychological safety.

Why You Can't Buy Cloud Native

Thu, 19 May 2022 00:00:00 GMT

Figuring out what problem we’re trying to solve
Releasing
Testing, but not the way you’re used to

Tradeoffs, Bad Science, and Polar Bears – The World of Java Optimisation

Fri, 13 May 2022 00:00:00 GMT

Present and Future of the Microservice Architecture

Fri, 13 May 2022 00:00:00 GMT

The panelists reflect on various microservices topics.

The Future of Java

Wed, 11 May 2022 00:00:00 GMT

Seven Ways to Fail at Microservices with Holly Cummins

Tue, 10 May 2022 00:00:00 GMT

Implementing microservices is really challenging, and there are many ways to fail. Holly Cummins has identified seven ways to fail at microservices and how they can be avoided.

SRE: The Good, The Bad, and the Ouch

Thu, 28 Apr 2022 00:00:00 GMT

SRE sounds like a plan with no drawbacks … but making it work in practice can be trickier than the theory says. This talk shares stories of SRE wins and SRE accidents.

Could new software tools make anyone a programmer?

Thu, 28 Apr 2022 00:00:00 GMT

The development of low-code and no-code systems makes it far easier to create new software, with benefits for both amateurs and professionals

No title

Thu, 14 Apr 2022 00:00:00 GMT

A Farewell to IBMs

Fri, 25 Mar 2022 00:00:00 GMT

Twenty one years ago I was heading into the Hampshire countryside for a job interview. I was a little graduate student, exploring possibilities in the world of work. I’d met folks from IBM Research at some academic conferences, and they were working on interesting things, seemed well-fed and happy, and had entertaining stories about goulash-based work mishaps. I didn’t have a computer science degree, but computers seemed fun, and IBM seemed like a good place to do computers. When I arrived at IBM Hursley, the first thing I saw was sheep on the front lawn. I was smitten.

The sheep:

Photo by Sam Carter on Unsplash … and if, like me, you adore sheep, go read this amazing article about sheep.

Me:

This is me looking serious and doing important quantum computing research for my doctorate.

I had facial piercings and pink hair, so it’s safe to say I didn’t look like most of the other candidates. I didn’t mind being different, but I didn’t want to look like I hadn’t made an effort. I wore a wool trouser suit I’d sewn myself, and I smartened up my pink hair with a fresh coat of dye. ‘Coat’ is an accurate word, because non-standard dyes don’t always stick very well. The IBM interview was a two day process, with an overnight stay in between. To my complete horror, in the morning of the second day of the interview I discovered that I’d stained all the pillowcases and towels fuschia. Realistically, IBM HR weren’t going to have difficulty identifying which pink-haired candidate did the damage.

Can you get hired if you trash the hotel room on the interview? Apparently, you can. And thus began my wonderful IBM career.

Pink hair wasn’t very IBM … so I fixed that by dyeing it blue to match the IBM corporate colours.

That photo was taken in this beautiful Hursley building:

As well as the sheep and the grade II listed 18th century mansion, IBM Hursley has a cricket wicket, a baseball diamond, a Japanese fish pond, and a museum. When I joined it also had a tiny in-person-bank and a dry cleaning service, but those have gone the way of the fax machines.

Hursley is gorgeous, and as the largest software development lab in Europe, it’s got an amazing tech legacy. The Spitfire was developed in Hursley during the second world war (not by IBM!). CICS, the software that runs almost every cash machine in the world was developed in Hursley. More generally, IBM has been responsible for so many world-changing innovations (bar codes, lasix surgery, electron microscopes, forty years of quantum computing). It combines ‘changing the world’ with a commitment to integrity and a humane corporate culture. Years ago someone from another large IT company asked me if I had any mentors - I explained that I had several, because senior IBMers are expected to grow those around them by actively mentoring. Mentoring is part of our performance assessments and promotion packages. Over the years, many IBMers have been very generous with their time to help me out. She was surprised, which surprised me. It turned out her company didn’t have any organisational incentives to lift others up. Employees were on their own when it came to growth, and usually found mentors from outside the company.

In contrast, IBM encourages growth and learning. As well as the mentoring, I’ve had lots of leadership training and coaching. I’ve learned C++, C#, Javascript, Ruby, Python, and TCL, along with UML, XML, TDD, XP. I’ve learned to draw, and written a book. IBM funded me to do an MSc in software engineering. I’ve travelled to five continents. I developed a bit of expertise in Java garbage collection, enough to end up on a billboard in Bangalore:

While we’re on the subject of billboards, I ended up on billboards again ten years later, although with more conventional hair:

If you find a job you love, you’ll never work a day in your life. I’ve made the best friends and had the best times at IBM.

I’m so grateful to IBM for taking a chance and hiring my despite my punk-rock interview destruction, and I’m grateful for all the support and opportunity I’ve received in my time here. It feels super-odd to be turning this in.

So, yeah. IBM: 10/10, would recommend.

(Wait, if IBM is so wonderful, why am I leaving? Well, I know it’s not the done thing to say where one is going in a goodbye message, but … let’s just say I’m not going that far.)

マイクロサービスに失敗する7つの方法

Sun, 27 Feb 2022 00:00:00 GMT

昨年 11 月の QCon Plus、私は、マイクロサービスがうまくいかない理由について講演しました。取り上げた問題は私の経験に基づくものであると同時に、残念なことに、現場で何度も目にするものでもあります。

简历驱动开发？微服务中的几种失败路径_文化

Sat, 26 Feb 2022 00:00:00 GMT

简历驱动开发？微服务中的几种失败路径本文要点：微服务是一种手段而非目标分布式并不能确保解耦性契约测试在所有微服务架构中都是重要组成部分前

GitOps in Regulated Industries Webinar

Wed, 23 Feb 2022 00:00:00 GMT

Ian Miell, Daniel Jones, Holly Cummins, and Jamie Dobson discuss the rewards and challenges of GitOps in regulated industries.

7 Façons D'échouer Dans Les Microservices

Fri, 18 Feb 2022 00:00:00 GMT

Lors de QCon Plus en novembre dernier, j’ai présenté certaines des façons dont les microservices peuvent mal tourner. Ces problèmes sont basés sur mon expérience que je vois souvent sur le terrain.

7 Ways to Fail at Microservices

Mon, 14 Feb 2022 00:00:00 GMT

At QCon Plus last November, I presented some of the ways microservices can go wrong. These problems are based on my experience – which, unfortunately, I see repeatedly in the field.

Is your innovation funnel an innovation fizzle?

Thu, 03 Feb 2022 00:00:00 GMT

World-changing ideas don’t seem normal and sensible – at first. They usally teeter somewhere between ‘impossible’ and ’ idiotic’. Take IBM’s Jeopardy-winning Watson system. I’ve seen a few different accounts of its creation, but I like the one shared with me by a friend who was there. An IBM VP, Charles Lickel, noticed everyone in a restaurant stand up and move to the the bar. What were they doing? What could be so interesting, in a restaurant bar? It turns out the bar had a TV, and they all wanted to watch the reigning champion of Jeopardy continue his winning streak. Lickel had an idea – IBM Research should build a computer which could play Jeopardy as well as a champion. His colleagues were more alarmed than impressed; the scientific consensus was that it was far too hard, and even trying verged on folly. Everyone knew that computers couldn’t handle puns, or humour, or allusions and indirection, much less all at the same time. There were mutterings that the old man had lost it. The happy ending to the story is that one scientist, David Ferrucci, stepped forward and said he thought he could do it – and succeeded. Victory was far from guaranteed, though: even once it was underway, the Watson project was subject to much ridicule from its competitors and even within IBM.

We see similar patterns of dismissal and ridicule for other significant inventions. When laptops were introduced, the New York Times questioned whether anybody would actually want to use a computer outside of work. They concluded ” the microcomputer industry has assumed that everyone would love to have a keyboard grafted on as an extension of their fingers. It just is not so.” Even cheeseburgers were dismissed as “a bizarre combination of beef and cheese” when they were introduced. This isn’t because the journalists were stupid, it was because, at the time, portable computers or melted-cheese-on-a-beef-patty genuinely seemed like terrible ideas. I can’t even comprehend not liking cheeseburgers, but there we are.

How does an organisation distinguish between truly idiotic ideas, and those which are just uncomfortably new? The standard answer is “build an MVP”. Sadly, as a term, MVP no longer means all that much. In 2022, it seems everything is an MVP. Some of these MVPs are experiments, some are prototypes, and some are just first releases. Some are £12 million “first releases”, developed by a waterfall process (true, sad, story). Because “MVP” can be used to mean very different things, when we say “MVP”, it’s important to be explicit about what we’re trying to accomplish.

What we should be trying to accomplish depends a lot on the context. (Cynefin can be a useful framework here.) In environments of high uncertainty, sinking £12 million into a first release before getting feedback is an atrocious idea. (I’m assuming there must be some environments where the uncertainty is low enough that spending £12 million before getting feedback is a good idea, although I haven’t to confess I haven’t encountered any of them myself.)

If uncertainty is high, we should start with an experiment. A small experiment. The focus here must be on learning, not on building or re-use. By definition, when we do an experiment, we don’t know what the answer will be. This means there needs to be a strong focus on metrics, and they have to be metrics which guide decisions, and not vanity metrics. We mustn’t formulate a hypothesis and then cherry-pick evidence which confirms our favoured theory. An experiment must be designed so that failure is possible, and possibly even desirable. A failed experiment is a good thing, as long as the experiment was small; we learned something quickly and cheaply which will prevent an expensive and slow mistake.

What happens if the results of an experiment aren’t what we expected (“failure”)? We adjust course. If the results are what we hoped for, we then go on to design a follow-on experiment which checks out our next risk. Eventually, if we do enough experiments, we will have moved from a situation of high uncertainty to one of moderate uncertainty.

At this point, experiments aren’t needed any more and we can shift our development technique to one which involves more, well … development. Our focus changes from learning to user value. What we’re delivering should be a product, something our audience can benefit from. That means we need to meet minimum standards for functionality (there has to be enough) and quality (bugs annoy users). “Minimum standard for functionality” doesn’t mean a product has to be feature complete. Users will tolerate missing features, especially if they can see that new capabilities are being rolled out regularly, and the product team is responsive to feedback. Here, “regularly” means “every few days” or “every week”, not “once a quarter”. Spacing releases out too infrequently can lead to value sitting on the shelf (bad) and half-baked features being rushed into a release to meet the deadline (worse). Continuous deployment is a key enabling technique here.

Getting a radically descoped product out into the field will allow development teams to prioritise features more effectively. Often, what we think our users want ends up being pretty different to what they actually want. You might be surprised how much can be descoped. The founder of Zappos famously descoped “actually having inventory” and “wholesaler relationships” when he started selling shoes. Instead, he would fulfill orders by going out to his local shoe shops and buying shoes. It’s not quite the same scale, but my team in the IBM Garage replaced a wholly manual ordering process with a web app … which sent an email to a seller under the covers. The client wasn’t sure if their users would feel comfortable ordering construction supplies online rather than over the phone, so the hollowed-out app allowed them to explore real-world user behaviour, and give real value to users, all for a modest investment. The app-to-email sales process was not particularly efficient on the fulfilment side, and it was not sustainable long-term, but that’s ok. Delivering a partial app allowed the team to get consumer buy-in and gather user feedback while continuing to build up function.

This type of MVP is known as a wizard of Oz MVP; it looks like a proper product, but under the surface there are smoke and mirrors and gophers pedaling frantically.

This idea of using experiments to prune out bad ideas early seems logical, but it’s something organisations can struggle with. In many cultures, it’s safe to say that “designing for failure” is not encouraged. Teams working on an idea have an incentive to prove that the original idea was great, and the implementation is successful. This means there is little motivation to build in the kinds of experiments and learning which could lead to a project being cancelled. This can cause an “innovation railroad”, where projects, once funded, are pushed right through until the end. This doesn’t mean the projects succeed, it just means failure is deferred and more resources are consumed before the failure is discovered. The sunk cost fallacy can also contribute to innovation railroads.

Another anti-pattern I see is the innovation fizzle. I heard recently about a government organisation who sent several dozen staff to a two day design thinking workshop. They were energised and creative and came up with pages of great innovation ideas. And then … nothing happened. Six months later, there was still nothing. Probably, not all of the ideas should have been built, but some of them should have been explored futher. In an innovation fizzle, ideas are generated, prototypes are written, annual appraisals are submitted describing all the innovation activity … but nothing makes it into the hands of real users. Although there is an innovation budget, sometimes a very healthy innovation budget, there doesn’t seem to be a meaningful path to production out from the innovation team. This isn’t laziness; it can be genuinely hard to push over the hump from “risk-free PoC” to “project living and breathing out in prod where users may get annoyed and security breaches may happen.” In an innovation fizzle, nobody makes a conscious decision not to proceed – it just … doesn’t happen.

Is there a healthy medium between “everything gets shipped no matter what” and “nothing gets shipped?” In principle, yes: the innovation funnel. Ideas are evaluated, the weak ideas fail-out fast, and the strong ideas continue on to production. But even properly funnel-shaped innovation funnels are not perfect. The evaluation stage of the funnel can be bureaucracy-heavy and focussed on error-prevention. For one industrial company, the process of investing in a new development took a hundred distinct steps. In others, teams spend significant admin effort working through approvals and generating polished proposals. Evaluation boards want to pick winners, and this can lead to them favouring safe bets rather than riskier, more innovative, ideas.

The focus of innovation gates should be on enriching ideas, and gathering real, external, feedback, rather than internal evaluations and paperwork. In part 1 of this blog, I told the story of George Washington Carver. Although Carver was an enormously successful inventor and educator, he didn’t get everything right. When he first started working with farmers in the south, he printed brochures advising them to buy a second horse so they could run a two-horse plough, allowing them to till the exhausted soil more deeply. While this undoubtedly would have improved cotton yields, it was hopelessly impractical advice for tenant farmers on the edge of starvation. Once he worked out the underpinning economic issues, Carver was able to switch tack and focus his educational efforts on composting (free!), swamp muck fertilizer ( squelchy, but free!), and crop rotation (sort-of-free).

Real field feedback is intuitively more accurate than paper-based or hypothetical feedback. Surprisingly, it can also be less expensive. Unilever launched four Peperami variations concurrently in 2004, in a sort of sell-off-sales-competition. In the end, three of the variations had disappointing sales and were cancelled, but this wasn’t a failure. Experimenting in the actual market reportedly cost £1 million less than Uniliver’s traditional, protracted, focus-group method of product evaluation. It also got the winning product, Peperami Firesticks, to market a year earlier than it would have otherwise done.

It’s a fact of life that innovation isn’t always going to work. Thomas Edison famously said “I have not failed 10,000 times—I’ve successfully found 10,000 ways that will not work.” Peperami Firesticks were a success, but Peperami Canniballs, Dunkers, and Sarni, launched at the same time, were not. Besides his tone-deaf horse suggestions, other notable Carver failures include peanut bread, peanut sausage, peanut coffee, peanut facecream, and peanut nitroglycerine. (They all “worked”, in some technical sense, but it turned out no one wanted peanut coffee in 1916.)

For innovation to flourish, these setbacks have to be tolerated. In fact, these setbacks have to be positively encouraged, as part of maximising learning. Innovation does not work in a culture of fear. Psychological safety is an essential foundation which allows teams to report failure and iterate continuously. The best part is that the ingredients for successful innovation - psychological safety, continuous delivery, short feedback cycles, and an emphasis on learning - are more generally the ingredients for a successful business. Everyone wins.

Why Your Innovation Team May Be Stifling Innovation, and Other Strategy Lessons From Peas

Mon, 24 Jan 2022 00:00:00 GMT

Innovation is great. I think so, I’m sure you think so, and I suspect we all think so. Innovation is good for business, and it’s good for us, and it’s healthy.

I like to think I’m kind of innovative; my job title is even “Innovation Leader”. I’ve written a mobile app to count fish using Watson, and I created the world’s first application server in a hat, and invented a cuddly throwable application server. (Remember, not all inventions change the world. Or are useful.)

Barriers to innovation

And yet … like many of us, I wish I innovated more. Why aren’t we all innovating as much as we would like? What’s holding us back? In every organisation I’ve worked with, the barriers to innovation are depressingly similar:

Time
Money
Ideas

I’m not going to write about how to come up with better ideas, because there’s a lot of great material out there. Let’s talk instead about how to make more time. The answer is surprisingly simple. All you need is …

… a time machine.

Of course, a time machine only fixes half the problem. You’re also going to need office space (well, in 2021 maybe not so much). And you’ll likely need cloud compute, or some cool bits of hardware, or external expertise. The way to secure those resources is …

… a money tree.

What if you don’t have a time machine or a money tree? Well then, sadly, you’re in the same boat as the rest of us. The good news is that there are ways to make time to innovate without defying the laws of physics or economics.

Strategies for making time

Techniques for building innovation into an organisation range from a centralised innovation department to just … allowing slack in the system for grassroots innovation. There are lots of options in the middle, too, such as hackathons and 20%-time projects.

Before we discuss the tradeoffs of the various strategies, I’d like to share a story. If you went to an American school, you were most likely taught about George Washington Carver, who invented peanut butter. In fact, Carver’s contributions to the science of ‘not starving’ were much more significant - and he actually didn’t invent peanut butter.

Carver was born into slavery in the American South, just before the Civil War. As if that wasn’t a rough enough start to life, as a baby he and his mother were abducted by slave raiders. His master managed to recover him, but, sadly, not his mother. When slavery was abolished Carver was raised as a son by his former master. However, educational prospects in his area were limited, so Carver travelled across the country, working his way through school and then university as he did so. Carver had a knack for plants, so he studied botany and went on to teach it.

Even after the abolition of slavery, life was bleak for African Americans in the South. Cotton is an unusually needy crop, and generations of cotton farming had stripped all nutrients from the soil. The survivors of slavery were left to try and scratch out a living growing cotton on soil which could not support them - or anyone. The land, and the people, were exhausted and hungry. It was hard to see a way out from the vicious cycle of poverty; expensive fertilizers which might have restored the soil were far beyond the reach of farmers who could barely feed themselves.

If you’re a keen gardener, you probably know that peas and other legumes can fix nitrogen from the air and enrich the soil. Despite the name, peanuts are related to beans, and also improve the land.

This is where Carver comes in. He discovered that by rotating cotton crops with peanuts, farmers were able to grow more cotton. Farmers loved the improved cotton yields, but they had a problem. Peanuts. What should they do with all of these peanuts they were producing?

At the time, peanuts were barely considered a crop, and were only used for animal feed. While people were going hungry, peanuts were being discarded as waste. This is Carver’s second great contribution. Carver pointed out that peanuts were high in nutrients, especially high in protein, and that they were also pretty tasty. He catalogued peanut recipes and invented many more, such as chocolate-covered peanuts. Carver’s work led to the adoption of peanuts as an ingredient, which persists in Southern cooking to this day. He also invented some surprising industrial uses for peanuts, such as peanut shampoo and peanut nitroglycerine. (A century earlier, Antoine-Augustin Parmentier was responsible for a similar effort to change the image of the potato. Parmentier’s work led to the decriminalization of potatoes in France, which just goes to show the history of food is stranger than you might imagine.)

Carver was a prolific inventor, and his innovations improved many lives, but I think there’s a deeper lesson in his story. The way peanuts can enrich soil while also yielding a useful crop is profoundly counter-intuitive. It’s like subtracting one from one and ending up with four. Bonkers.

Do we see these sort of ‘should be subtractive but are additive’ effects anywhere else? I believe we see them in people. If you ask a team to do the same thing, over and over again, they get tired. They burn out. Productivity suffers.

Rest helps revive tired teams, but sometimes a change is as good as a rest. When a team is following its passion and innovating, they’re inspired. They get a bounce in their step; their eyes sparkle and they stay late just because.

This is why setting up a dedicated innovation teams can sometimes miss out on the full potential of innovation activities. Siloed innovation teams are a bit like repeatedly growing cotton on one field, and repeatedly growing peanuts on a neighbouring field. The innovation team will be having fun, but they may miss out on the experience of real world challenges which would lead to more meaningful innovations. Meanwhile, the ‘business as usual’ team risk being ground down by too much work and too little play. Innovation, particularly Big Innovation, needs to be managed differently and funded differently, so innovation teams play an important part in an organisation’s innovation portfolio – but they shouldn’t be the only entry.

20% projects

Many businesses recognise the value of rotation to spread the innovation fun, and have adopted different strategies to support part-time passion projects. For example, Google famously has a 20% rule which allows employees to spend one day a week on projects of their choosing. Many Google flagship products, such as GMail, Google Maps, and Adsense, originated from this part-time employee tinkering.

There’s a problem, though. Often, takeup is low, and the program ends up being a headline, rather than a reality. This isn’t because of leadership malice; it’s just that there is always more work to do than there is time. Our day jobs expand to fill the contracted working hours, and then usually expand beyond them. We’re bogged down by day-jobness.

Hackathons

To counteract this, some organisations organise hackathons. The idea is that employees go somewhere, eat junk food, drink too many caffeinated beverages, stay up late, sleep on the floor, and build exciting prototypes. The problem is that hackathons almost always happen on the weekend. The time shortage is being ‘fixed’ by taking a chunk out of employee’s personal time.

More generally, employee ‘passion’ can be used as a means of extracting extra labour from the workforce, without extra compensation. Loving one’s job is good, but it shouldn’t be a lever to enable exploitation.

Off-plan post-release weeks

“Off-plan post-release weeks” are one of my favourite innovation mechanisms. The name isn’t great, because I made it up. The origin is an experience I had in the WebSphere product team, many years ago. Back when we only released infrequently, releases were a big big deal. There was always a bit of a rush to meet the release deadline and get as many features as possible onto the train before it left. Here, ‘bit of a rush’ is a nice way of saying ‘increasingly stressful period of evening and weekend-working’ (this is a great reason to make releases more frequent, but that’s a separate topic).

Once the release was out the door, there’d be a collective flop and let down of adrenaline among the development team. To make up for desperate weekend-working grinds, we’d get all time off. (Have I mentioned that more frequent releases are a great idea with many efficiency benefits?) If the release had just been mildly exhausting, we’d get a different kind of treat. All plans and schedules and backlogs would be shelved for a sprint, and the team could work on whatever product features they liked.

It sounds like a rough deal that we after working really hard our reward was … the opportunity to do work. But it was great! The team loved it, and some of our best features came out of those off-plan weeks. What makes this model stickier than 20% time is that the whole team took it at the same time. Day-to-day commitments couldn’t derail the innovation work, because they all went away.

The success of “free time to do work” as a reward shows the importance of autonomy; allowing teams to decide what to build, even for just a sprint, made people happy and unleashed a flood of innovation.

Slack in the system

What if innovation could happen without any 20% rules, or post-deadline-exhaustion, or junk food and Coca-cola? What if we just … had enough space to come up with great ideas, and then had enough time to try them out?

Well, it’s a bit utopian. Most of our workplaces aren’t so luxurious. But that doesn’t mean we shouldn’t strive for space. There’s a growing body of evidence that healthy workspaces, with generative cultures and happy employees, are associated with higher business performance. An over-emphasis on productivity metrics, like other metrics, could actually be lowering productivity. Even five day weeks might be lowering productivity.

What does this mean in practice? As leaders, we need to re-evaluate how reliant we are on superficial productivity metrics, and shift to thinking about outcomes instead. What might look like a spontaneous employee ball-game involving maltesers, comedy hats, and wastepaper baskets, could actually be an employee wellness experience which is getting blood circulation going, improving employee health, and averting (expensive) staff burnout. What might look like goofing off or daydreaming could actually be laying the foundations for a breathrough innovation.

In the next part of this blog, I’ll talk about choosing innovation winners, innovation funnels and innovation fizzles and the perils of MVPs.

(Developer) Joy to the World

Tue, 21 Dec 2021 00:00:00 GMT

I love this time of year. There are so many celebrations, both new (Kwanzaa) and ancient (Winter Solstice). My friends’ timelines become a happy jumble of Christmas, Hanukkah, and the New Year. Imprinted in our modern traditions we can still see the shadow of older holidays, such as Yule and Saturnalia. The appeal of celebration is so strong that even countries with no religious connection to Christmas get in on the Yule fun with showpiece trees and ( slightly baffling) Kentucky Fried Chicken rituals.

As the lights go up and I fill my house with fir, holly, and mincemeat, I feel fulfilled and content. Mostly. There are some useful lessons for software development from our approach to the winter holidays. Or maybe it’s the other way round, and we can get better at the holidays by importing some software development practices.

Enabling constraints

Why do so many joyous holidays happen in what is, arguably, the worst time of year? December is cold, wet, and dark. In my part of England the damp gets into your bones and the sun sets at three in the afternoon. Celebrating in this miserable environment isn’t a ‘despite’, it’s a ‘because’.

Many of our Christmastide customs are imported from northerly countries, where if anything it’s even colder and darker. Our winter traditions are about the joy of light in the darkness; fairy lights, candles, and illuminated garden displays. We bring the light not just into our houses but into public spaces, so it can be shared. On my run this morning, I noticed that someone had done some joyful civic-minded guerilla decorating and hung baubles on the trees in my local arboretum. No one does that in the summer.

The dark and cold is an example of an enabling constraint. Our response is to turn a defect (cold, dark, brr) into a feature (twinkling lights everywhere! mulled wine! thick socks! hygge!) The apparently unpleasant external conditions allow a flourishing of creativity.

Why we misunderstand efficiency

Every December, I go through the same cycle. In the run-up to Christmas, I get more and more excited. I start making lists of all the things I’m going to fit in this year - Christmas cards for all my acquaintances with long, detailed, personalised messages! Homemade Christmas cakes with homemade marzipan! From-scratch mince pies! Cranberry and pistachio florentines and vanillekipferl for all my office-mates and neighbours! Carefully selected, thoughtful, touching gifts for everyone on my list! Carol services! Skating! A well-architected gingerbread house! This will be the year I really master Christmas floristry! Meaningful family moments!

And then there’s the house. Who doesn’t want an insta-worthy home at the holidays? Yesterday I spent £130 on LED pillar candles to try and make my staircase look like a photo I saw in a magazine. (Spoiler: it’s still not going to look like that picture and no one in the household is quite sure where these candles are going to be stored the rest of the year.)

Sometime around mid-December my partner finds me in tears on the kitchen floor because I haven’t sent a single card, I’ve missed the last posting date for Canada, my attempt at a hand-crafted fir garland looks like a rat king, I don’t have any gift ideas for anyone which means everyone will be disappointed with me on Christmas morning, and it took me a whole evening to half-finish one small batch of cookies and there’s six more batches to go and I don’t even know how to make Turtles. Oh, and I’m still catching up on the bulb-planting I should have done in the autumn and also trying to do some coding in the evening and I have a work deadline. Sustainable pace got abandoned somewhere in November. The ritual Christmas meltdown is as drearily predictable as schedule over-runs on software projects.

Every year I tell myself that if I’m properly efficient, I can fit it all in. But efficiency isn’t doing more with less, efficiency is just doing less. This is the principle behind lean software development and the MVP.

But doing less is hard. It’s really hard. Somehow, I haven’t learned the lessons from my annual Christmas meltdown. Doing less means giving up on perfection. It means living with defects and missing features. Organisations find this really hard, too. When I was a consultant in the IBM Garage, de-scoping MVPs to be genuinely-minimal was one of the hardest parts of my job. As an industry we haven’t really got the hang of success metrics that aren’t focussed on doing as much as possible.

The value of rest

This is a shame, because a growing body of evidence suggests that long hours and piling on the pressure actually hurts productivity. In A Christmas Carol, Bob Cratchit famously worked sixty hour weeks (the standard at the time) in miserable conditions. Working hours have been steadily reducing since the end of the nineteenth century, as employers figured out that if their employees worked shorter hours, they would be more productive. This was empirically validated by British wartime research in munitions factories, which found productivity declined sharply above 50 hours a week. Above 55 hours, productivity dropped so much that the extra hours were effectively wasted.

Of course, most of us don’t work in industrial settings, and our output is harder to measure. But knowledge worker’s hours have been steadily creeping up. That trend is now at least partially reversing, with some companies adopting a four day week. Paying staff the same for working fewer hours seems bonkers, but the evidence seems to be that productivity stays the same or even goes up.

The value of joy

A crushing grind hurts employee morale, and that affects outcomes. If a work environment is positive, on the other hand, there will be fewer sick days, fewer costly errors, and staff will actually work harder. The business will do better. The same principles apply even to individuals; one researcher found people with a positive mindset are 31% more productive than those with a negative mindset – or even a neutral one.

I’ve written and spoken about this a lot, arguing for fun in the workplace. I chose to talk about fun rather than happiness or positivity because it’s less obvious that fun at work is good; our natural Calvinist assumption is that happiness might maybe be a bit ok but fun is morally dubious and certainly not the sort of thing we should indulge in at work. Incidentally, the same sort of Calvinist thinking led to Christmas being outlawed in Scotland in 1583, England in 1647, and Boston in 1659. Christmas wasn’t recognised as an official public holiday in Oklahoma until 1907. Scotland waited until 1958. The root of the objection was just that people had too much fun on Christmas.

At Christmas, actually, we go beyond mere fun, and we strive for joy. Just like fun, joy at work is good. Think about that amazing feeling of finally finally finally figuring out a thorny bug: what you’re feeling is joy. Some businesses place so much emphasis on employee joy as a strategic advantage they’ve encoded it into their corporate identity.

There are a lot of words for ‘good feelings’. I sometimes use them interchangeably, but there are nuances and a range of ‘exactly how good do we feel?’ and ‘how much would Calvin object?’ values. I’ve tried to untangle some of them below. My categorisation of ‘perceived utility’ is pretty unscientific, but the important thing here is that there are two axes, not just one. Being useful and feeling good is a (possible) and, not an or. I did include the opposites, but not too many of them, because it’s Christmas, and who wants to look at a big list of miserable things?

Joy and fun are good in corporate cultures, and they’re also good in products. I loved seeing IBM’s Code Engine introduced with an article inviting developers to “enjoy your cloud again”. Quarkus have developer joy as a core requirement. James Governor has been pointing out for a while that prioritising developer experience (DX) is good business sense. Frankly, some of the tools we have to use every day are definitely not frictionless, so I’m not sure everyone is listening - but it’s good to see momentum picking up in this area.

I hope you have a wonderful holiday, however you celebrate; and I hope you find light in the darkness and joy in your middleware.

What How Why - Quantum explained

Thu, 16 Dec 2021 00:00:00 GMT

IBM is working with businesses embracing new challenges and part of this investment is focused on high quality STEM education.

How this site is built

Mon, 15 Nov 2021 00:00:00 GMT

A long time ago, I used to write semi-regularly on Blogger. But then I stopped. What was up? Was it that I wasn’t making time to write? Was it that I needed to form better habits? No. Clearly, the problem was that I needed a newer, shinier blogging platform.

I’d been inspired to think about self-hosting by Trisha Gee’s switch from Blogger to Hugo. But of course I couldn’t just create a Hugo site. Instead, I had to investigate lots and lots of various tech stack options.

In my defence, there was a seven year gap between when I read Trisha’s post in 2014 and when I actually took the idea to action. I’m slow. So it wasn’t unreasonable to assume that the static site world might have moved on while I was busy being slow. (It turns even Trisha’s moved on from Hugo, to Wordpress.) Trisha’s site rebuild notwithstanding, Hugo is still a popular static site framework. But not the only one:

I’m a coder. Of course I wasn’t going to use an antiquated, simple setup, even if it was actually very productive. I needed continuous integration! And complex dynamic behaviour! The leading static site generators were Hugo, written in Go, and Gatsby, written in React. I need to improve my Go, and building a Go blog would have been a great way to do that. I resisted temptation and decided to (sort of) prioritise productivity by going for Gatsby.

It took me a while to settle on a platform, and then a while to find the right starter, and then I spent several weeks adjusting the starter to support unit tests, and setting up a CI/CD pipeline with integration tests. I didn’t quite get to the point of setting up a Kubernetes cluster to run my static blog, but I wasn’t far off.

Once I had a unit test skeleton in place I felt able to start adding new capabilities to the platform. A lot of my writing is published elsewhere, and some of my content isn’t even writing, it’s slides and podcasts and external media. (I got profiled in Forbes! That has to go in somewhere!) So I needed a hybrid between a blog and a portfolio site. I wasn’t just looking for an excuse to code, honest.

The tech black hole

Now that things are up and running, I’m trying to balance time spent on writing and time spent on improvements to the base platform. I wasn’t going to be That Developer:

I was also determined that the things I wrote about were going to be things other than my blog tech setup.

No no, no snake-eating-its-tail blogging for me!

And I was going to write actual posts about actual things!

If anything, I’ve probably done too good a job of focussing on content in the past year, at the expense of sorting out the platform capabilities. I still need to figure out the right solution for slides. At the moment they’re hosted separately, and I pay for that. They’re not even cross-linked into the main content feed. (I like Notist, but the price of Notist Pro pains me, because I am a cheapskate.) Several people have asked for RSS feeds, so I need to sort that out. And my search is a bit busted for external links, so I need to change what gets fed to the indexer.

So if I wasn’t going to write about my tech stack, why have I just written an article about my tech stack? Well, um, people asked. And, you know, I worked hard on it and wanted to share.

Ok but what is the stack?

The source code for this platform lives at https://github.com/holly-cummins/gatsby-platform.

This blog is a GatsbyJS blog, based on Greg Lobinski’s HeroBlog starter. I’ve extended it to make it easy to embed external content and to add different forms of content. There are microanalytics.io Analytics and Algolia search integrations. The README has getting started instructions and more technical details.

I pull some graphics from unsplash and draw others. A lot of the visual design is the same as in the starter.

Testing

I’ve got the TDD habit, so it made me feel a bit itchy that all the gatsby code I was looking at was test-free. I added some Jest unit tests, and also some puppeteer integration tests. It would benefit from more tests, but at least it has some now.

Hosting

I use namecheap.com for my domain. For a while I had hollycummins.fun redirecting to this site, although I never publicised that awesome URL, so I let it lapse. (Remember, fun in the workplace is important.)

I use GitHub Pages for the hosting and GitHub Actions for the CI/CD. It’s free, and I am a cheapskate.

Work tracking

I’m using Apple Reminders as my work tracking system, but I should probably switch to GitHub issues.

Ok but can I re-use it?

The platform code for this blog shares the MIT license of the source starter. If you use my fork I’d love to know about it. You can see what the starter looks like without my content at gatsby-platform.hollycummins.com.

Most of the personalisation is in the content folder, but there are a few bits and pieces elsewhere, like avatar.jpg and src/images/jpg.

The content itself is not open source.

A Journey to Automation from Observability to Sustainability

Thu, 04 Nov 2021 00:00:00 GMT

Why I never want to build another MVP

Tue, 02 Nov 2021 00:00:00 GMT

Holly Cummins, Innovation Leader at IBM, discusses the benefits and challenges of building minimum viable products (MVPs).

Software Engineering Unlocked: Driving innovation and engineering practices

Tue, 28 Sep 2021 00:00:00 GMT

In this episode, I talk to Dr. Holly Cummins. Holly was the development practice lead for IBM Garage for Cloud, before becoming an innovation leader in IBM’s corporate strategy team. She drives innovation for companies in various industries, such as banking, catering, retail, or even nonprofit organization. She is also a Java Champion, a JavaOne Rockstar,…

Nine ways to fail at cloud native and how to avoid them

Tue, 31 Aug 2021 00:00:00 GMT

Cloud native is a perfect recipe for innovation, adaptability, and engineering excellence … when it goes right. Dr. Holly Cummins outlines potential cloud native disasters and how she and her team can help you avoid them.

DevOps and Cloud InfoQ Trends Report - July 2021

Mon, 19 Jul 2021 00:00:00 GMT

This article summarizes how we see the “cloud computing and DevOps” space in 2021.

WTF is Cloud Native Quantum?

Mon, 05 Jul 2021 00:00:00 GMT

Quantum computers represent an extraordinary feat of engineering. Dr Holly Cummins explains their potential, how they work, why they are such a great fit for the cloud, and what they have to do with cats.

Cloud Tech Thursdays: How To Love Kubernetes and Not Wreck The Planet

Thu, 10 Jun 2021 00:00:00 GMT

Cloud Tech Thursday explores the full modern open source cloud stack, from hardware to serverless. Learn about new ideas, projects, and releases around Kube…

In the Open with Luke and Joe | Holly Cummins | Innovation Leader (a podcast, but with pictures)

Fri, 14 May 2021 00:00:00 GMT

Please join us for a conversation with IBM Innovation Leader Holly Cummins.

WTF Does Tech Have to Do With the Planet?

Fri, 30 Apr 2021 00:00:00 GMT

Our industry consumes around 1% or 2% of the world’s energy. If you consider the broader IT industry, our energy usage is higher still.

The InfoQ Podcast: Software Architecture and Design InfoQ Trends Report—April 2021

Mon, 19 Apr 2021 00:00:00 GMT

An overview of how the InfoQ editorial team sees the Software Architecture and Design topic evolving in 2021, with a focus on what architects are designing for today.

Software Architecture and Design InfoQ Trends Report—April 2021

Mon, 19 Apr 2021 00:00:00 GMT

An overview of how the InfoQ editorial team sees the Software Architecture and Design topic evolving in 2021, with a focus on what architects are designing for today.

Serverless doesn't have servers, except when it does

Thu, 15 Apr 2021 00:00:00 GMT

There are only two hard things in Computer Science: cache invalidation and naming things.

— Phil Karlton

Photo by Boba Jovanovic on Unsplash.

Naming things is hard, and bad names make developers unhappy.

Some things are easier to mis-name than others. Let’s talk about the naming challenges around serverless. For a while, we stopped calling it ‘serverless’ and started calling it ‘functions as a service’, or FaaS. I don’t have evidence, but I assume that everyone just got tired of people piping up with “but actually serverless does have servers, you know” every time anyone said “serverless”.

But there was a problem. For some use cases, running a function is fine. There’s still a server, of course, but the cloud provider takes care of defining it and provisioning it. But these free-floating functions aren’t so great for running locally. They’re always going to be tied to some execution framework, like Amazon Lambda or OpenWhisk. On the cloud, the user experience is frictionless. But to run the function on your machine, you either need to run the whole framework, or you need to roll your own lightweight harness to stand your little functions up and drive them. If your code has dependencies, that also gets awkward. Most FaaS frameworks have some packaging syntax for expressing dependencies and other metadata, but there’s usually a limit on how big those dependencies can be, and if you want to switch framework, you may need to rewrite all your metadata.

Managing free-standing functions just starts to feel like a lot of work. Why insist on trying to make functions, when we have containers? They’re light, portable, totally encapsulated, and ubiquitous. Containers were so great for this use case, some FaaS frameworks retrofitted the ability to run code in containers as well as loose functions. Then Knative appeared on the scene, and it could only run containers.

The problem? What do we name our new way of doing things? Every container file explicitly declares and manages a server. For example, I’ve just written this in a Dockerfile for a demo I’m doing on Code Engine, a managed Knative:

CMD [ "node", "server.js" ]

That’s a server! And I decide what it looks like and how it runs, so I own it. Serverless is a ridiculous name for something which has a server, if the user is the one managing the server. Serverless-but-with-a-server doesn’t roll off the tongue. Function as a service makes no sense, because it’s not a function.

In this layer diagram, grey parts of the stack are controlled by the provider, and blue parts are controlled by the developer. In both cases, there are servers in the stack, but in the model where containers are the units of deployment, the user controls them.

One name which got floated for this model was ‘funtainer’. This was pre-Knative, and waaay before last year’s Amazon Lambda container-support news. At the time, it was OpenWhisk, Spotinst, and OpenFaaS. Funtainer isn’t ‘a container which is lots of fun’, it’s a mash-up of ‘function’ and ‘container’.

But there’s a new problem. SEO.

Try googling for ‘funtainer’. You’ll get a lot of cheerful-looking Thermoses, and nothing about the cloud architectures. Apparently ‘funtainer’ is Thermos’s fun-container (as in, actual fun, or as much actual fun as you can have with an insulated drink container).

No problem, you can refine the search. What about ‘cloud funtainer’? You get a Thermos with a picture of a cloud on it.

And this is why Knative describes itself as ‘serverless’.

Another way of describing serverless, which involves less head-warping ‘there are no servers visible to the developer, but there are servers underneath, and actually the developer sees and defines those servers’, is this: “Billing per request rather than per instance”.

But there’s still a problem, even with that name. Well, the first problem is that it’s a bit of a mouthful (BPRRPI, anyone?), and the second problem is that it’s not even accurate. A managed Knative, like Code Engine, has billing, and it’s per request. So we’re good there. But if you run Knative in your own cluster, what you’re paying for is the cluster instance. Unless you’re really really good at FinOps, requests won’t be billed. Knative itself doesn’t ship with any billing support, so any name involving ‘billing’ is a non-starter.

What about ‘Container as a Service’? It’s a good name — but it’s been taken. Container as a Service describes a general managed container platform with per-instance billing. For example, Kubernetes could be described as CaaS-ish. (OpenFaas, which is all about running containers, not loose functions, is still a FaaS, not a CaaS. Did I mention all this naming stuff was hard?)

And this is why Knative still describes itself as ‘serverless’.

When TDD gets hard

Wed, 14 Apr 2021 00:00:00 GMT

Test-driven development (TDD) is a core IBM Garage practice. It provides the foundation for the other practices such as continuous delivery, DevOps, and automation. When you work in a test-driven…

Scaling your enterprise innovation with start-up speed

Thu, 01 Apr 2021 00:00:00 GMT

Practical strategies to accelerate the delivery of your future growth initiatives A collaborative chat between Sabine VanderLinden – CEO & Managing Director – Alchemy Crew and Holly Cummins – STSM Innovation Leader – IBM Garage We do not have a choice. Our world must become comfortable living with a heightened level of uncertainty. Learning from the recent crisis has helped businesses digitise faster than many others have in the past. Still, we now need to digitise consistently with internal and external experts eager to engage, learn and execute together really cool solutions and solve some of the most pressing societal problems we have yet to encounter. Both Holly Cummins, Innovation Leader and strategist within the IBM Garage and I had t

Cloud-Native Is about Culture, Not Containers

Wed, 17 Mar 2021 00:00:00 GMT

At QCon London last year, Holly Cummins, innovation leader in IBM Corporate Strategy provided a session titled: Cloud-Native is about Culture, not Containers.

What do fashion and programming have to do with one another?

Wed, 10 Mar 2021 00:00:00 GMT

Last night I dreamt that Roger Daltry’s wife told me to write about fashion and programming. We were in a small skate-punk shop. It was never made clear how she knew me or why she cared.

Photo by Flaunter on Unsplash.

By the end of the dream I’d almost written a “why fashion and programming are actually connected” blog, so I woke and made frantic notes to try and remember it before it slipped away. Although on the surface they seem pretty distinct, there are actually some interesting links between fashion and programming - or at least, I think so. Let’s talk about gender, creativity, labour ethics, climate, and trends - all in the same blog.

Gender

Fashion is strongly associated with women, and being a woman in tech can be uncomfortable. One of my coping strategies has been to not be excessively girly. Talking about girl-topics like fashion makes me feel vulnerable, so this may be the most personal blog I’ve ever written. (I nearly threw the whole thing away several times because I was so nervous about outing myself as GIRLY and being perceived as vapid).

I love being a woman, and I love working in tech, and I’ve been lucky that I haven’t had too many problems with the combination of the two. One of my coping strategies is that I try keep a strict separation between my work side and my girly side. I have a set of ‘work’ clothes’ and ‘home’ clothes, and they live in different chests of drawers, on opposite sides of the room. At home I have fun with skirts, sometimes quite short ones, but when I’m at work, I wear loose fitting tops, and always, always, trousers.

Honestly, I don’t think anyone’s actually fooled. A few years ago when I had booth duty at a conference, I was the only one on our work slack channel who asked what colour the booth t-shirts would be so I could pack coordinating eyeliner. I just like high heels and makeup too much, so I’m probably not blending in as much as I imagine.

I had pink hair when I interviewed at IBM, and blue hair for many years. My dad points out that pink hair is definitely not blending in at all, but my crayola locks felt punky, not girly. This is me, just before my interview with IBM, doing a quantum computing experiment and being an unwitting-stock-photo for the BBC.

I have such admiration for people like Trisha Gee, who are brave enough to bring their whole short-skirted self to work, and who write thoughtfully about it. Folks like Chloe Condon and April Speight go further and directly link their coding and fashion interests in their public persona. Check out April’s twitter handle.

For Chloe, very-pink-and-very-technical is part of her brand.

While it may feel easier at the time, women blending in with the boys doesn’t really help our industry. We should be visible as developers, and women, and hammering the #ilooklikeadeveloper hashtag, until we change the perception of what a developer looks like.

Creativity

Gender politics aside, programming and fashion actually have a lot in common. The reason I love both of them is that they’re creative. I get annoyed when I’m on a team with designers, and someone calls the designers ‘creatives’ to distinguish them from the lumpen developers. I’m creating too!

Similarly, sewing is much more mathematical than is sometimes acknowledged. I sometimes call sewing ‘textile engineering’ when I’m trying to make a point. I used to sew a lot of my own clothes, before I switched to using the same skills to sew IoT demos that looked like cuddly versions of product logos. It’s a niche hobby, but someone has to do it.

(In case you’re wondering, there was definitely some tough engineering in there, and resiliency was an ongoing challenge. I never did get the connections durable enough, or find a good mechanism for heat dissipation. It turns out computers don’t like being encased in padded balls. Who knew? )

Although it has its own gender problems, I love the vibrant female presence in the maker community, such as Limor Fried, Naomi Wu, and Rachel Konichiwakitty. Not all of them are focussed on wearables - Helen Leigh does amazing things with musical instruments - but clothes do provide an on-ramp to making for many. One of my IBM colleagues sewed her own wedding dress and made sure to include a generous sprinkling of LEDs, internet connectivity, and other tech gadgetry. Even my employer has got in on the fun and collaborated on a ‘dress that thinks’ for the Met Gala.

Even if they’re not behind the sewing machine, developers can find joy in uniting their professional and sartorial creativity:

The Shenova code dress (sadly, no longer available) could be printed with the wearer’s own source code, and ensured women speakers did not blend-in-with-the-boys on several conference stages.

Labour ethics

One of the reasons I liked sewing my own clothes, is I knew they were produced under humane conditions, and the worker ( me!) was treated well. The garment industry has some of the lowest pay in the world, and life-threatening accidents are not uncommon. Programmers, on the other hand, are are well-paid and are extremely unlikely to die at work. Nonetheless, labour exploitation is a concern, even for developers. If they had the misfortune to create a popular open source project, individuals sometimes find themselves expected to work for free for the benefit of profit-making corporations.

Even when there’s a clear compensation system in place, tech workers are increasingly pointing out that the conditions of their employment leave something to be desired. Although the pay is good, it’s not always fairly distributed to women and minority ethnic groups. There have been troubling problems with how sexual harassment cases are handled, and employee dismay over ethically dubious contracts. As a result of all this, unionization efforts are gaining momentum in the tech industry.

Sustainability

Both the fashion industry and tech industry have come under fire for their environmental impact. If you’re keeping track, the tech industry is responsible for around 2% of carbon emissions, and the clothes industry accounts for 10%. Many clothes are used once and then thrown out or forgotten in the back of a closet. This is bad, but it’s perhaps better than something like Grimes’ software-enabled NFT artwork, which I’m not convinced will actually be used at all. Its main value is its scarcity, as a collectible. Because NFT artefacts are dependent on a blockchain to ensure that scarcity, their electricity requirements are queasy-making. The production of 303 editions of a single Grimes NFT artwork is estimated to have produced 70 tonnes of carbon, and used the same amount of electricity as an average EU resident would in 33 years. With both clothes and NFT artefacts, the root problem is a culture which encourages conspicuously wasteful consumption.

Trends

The sustainability problem is a symptom of a social effect. We want to impress the people around us, whether that’s with our clothes, our art, or our work. One of the best ways to impress others is by showing them things they haven’t seen before. This continuous need for novelty drives cyclical tastes, both in hemlines and in software architecture.

Coco Chanel said “Fashion is architecture,” although I’m taking her slightly out of context, since she wasn’t thinking of microservices or edge computing. We all know the Gartner hype cycle, which shows how a technology will start out as ‘edgy and cool’, and then become trendy, and then mainstream, and then passé. Many observers have noted that microservices are closely related to SOA, which were the height of fashion twenty years earlier, before they fell out favour fifteen years ago. At the moment, microservices are so popular they’re considered near-compulsory for new software projects, and used in many contexts where they’re not appropriate.

There’s a similar cyclical pattern in our industry’s swings from centralised computing in the 1960s (timeshare) to distributed (mini-computers), and then back to centralised (client-server), and then back to distributed in the 1980s (PCs), and then on to centralised ( cloud), and then back distributed (edge computing). Dizzy yet?

In some cases, when a new pattern gains traction, it’s because something external changed to enable it (like how widespread fast network access created the right conditions for the cloud, or lightweight containers created the right conditions for microservices). Chanel had a saying for this, too - she insisted “Fashion is not something that exists in dresses only. Fashion is in the sky, in the street, fashion has to do with ideas, the way we live, what is happening.” Even in clothes, styles were heavily influenced by technological innovations. Lightweight steel for crinolines enabled the huge skirts of the mid-Victorian era. A few decades later, the discovery that brightly coloured dye could be extracted from coal tar enabled “mauve mania”, a fashion for the newly-possible lurid-purple clothes.

So there we have it. Microservices are the new purple, and programming can be pink.

November 2022: I’ve written a follow-up piece, “The Sad Case of the Tech T-shirt”.

All of this blog is speaking for me, and not my employer, but since I mention them in this blog, I’ll restate - these are my personal opinions.

IBM Developer Lead: What Software Does Next

Thu, 18 Feb 2021 00:00:00 GMT

Working to try and anticipate what comes next, how to harness current programming methodologies and build software for tomorrow and the day after is Dr Holly Cummins in her role as worldwide development community practice lead for the IBM Garage.

Coronavirus: Managing software development during lockdown

Wed, 17 Feb 2021 00:00:00 GMT

The Computer Weekly Developer Network (CWDN) examines the impact of the Covid-19 coronavirus on the software application development community.

The great microphone face-off, or 'how I spent lots of money in 2020'

Thu, 11 Feb 2021 00:00:00 GMT

The past year saw an unprecedented flourishing of “photos of home offices,” “recommendation threads for microphones,” and “microphone review videos with huge numbers of views.” I’ve experimented with a few different setups for talks, and I’ve been having to watch my own videos more than I usually do.

The usual “cringeing at myself” was compounded by “cringeing at my bad sound.” Since I’ve had a chance to accidentally-compare different microphones, here is an accidental-review.

The tests

Here’s what I’ve tried this year:

Sennheiser Game One

We use these headsets in the IBM Garage for remote pair-programming. Because the headset mic only picks up noise right near the microphone, typing is almost inaudible. This is a must for pairing, where people need to type and talk simultaneously and definitely don’t want to be always fussing around with mute buttons. These headphones also very comfortable, even for long periods (another must for pairing). One of my favourite features is a mechanical mute, so to mute you can just lift the mic boom. This is usually way easier to activate and de-activate than an on-screen control, and it’s way more obvious when you’re muted. When I listen back to recordings of me talking on conference calls, the sound is rich and natural.

All in all, I love this headset … or at least I did, until I started listening back to my presentations. If I get animated and start projecting my voice, having the mic right next to the mouth becomes a liability. All the ‘p’s and ’ t’s turn into big PPPBOOM sounds, known as plosives. Even breathing sounds can become intrusive.

You can hear the badness in my QCon Plus talk. I can’t link to particular times, but have a listen around 4:40, for example.

An IBM Fellow told me the headset look dorky, but I’m ignoring him. I think the big over-ear headphones and mouthpiece have got a kind of helicopter-pilot-cool to them. If you’re also into helicopter-chic, you’re not doing formal presentations or speaking while standing, and if you’re willing to make a bit of an investment (at the time of writing, £158), I’d totally recommend this headset.

Fifine 669B

At the other end of the spectrum, this is a relatively economical desk-top microphone. It’s plug and play, looks quite pretty, and will pick you up from some distance away, so there’s no awkward hunching over the mic. At around £48, it’s good value for money. However, of the three mics I tried, it’s got the worst sound. There’s a coldness to the sound, which might be caused by ambient echo from the room.

You can hear this mic in my Software Circus Hallowe’en talk. (Please excuse the costume, it made total sense at the end of October.)

Although I liked the price of the Fifine, I wasn’t happy with the sound. Judging from my twitter feed, absolutely everyone had already bought themselves a fancy boom mic, so I decided to Invest.

Røde PodMic

My current set up is Røde Podmic. At £99, it would be the mid-priced option of these three, except that it needs a lot of supporting equipment. There’s no stand, so I got a Røde boom arm to allow me to move it around. The Podmic uses XLR connections, so I had to buy XLR cables. I imagine sound engineers just have them lying around, but I certainly didn’t. It needs a preamp to convert the analog XLR signal into a computer-friendly USB-signal. Because I wanted to make sure there wouldn’t be any plosives, I bought a Røde windshield. Oh, and I needed separate headphones to make sure I couldn’t causing echo for anyone else on a call.

At this point, I thought I was done, but I wasn’t. It turns out I speak quietly. My sophisticated semi-pro sound setup meant … I couldn’t be heard on calls. The only way to be loud enough was to put myself an inch away from the microphone. I didn’t want to be right up against the microphone while I was talking, because that looks ridiculous on a video call.

Another £40 or so later, I had a booster amp and a second XLR cable, and I could be heard. I can even be heard with the mic fully off camera, although the sound is better if it’s a bit closer. So was it worth it? Well, you can decide, but I think there’s a clear difference between the sound in my WTFinar and Software Circus a month earlier.

In hindsight, the Røde Podcaster would probably have been more plug and play, with comparable sound. I’m not going to buy one to find out, though!

But what about you?

Clearly, not everyone needs an ostentatious boom-arm mic with weird specialty cables. Arguably, even I don’t need one. Nonetheless, if you’re doing video calls with other people, good equipment, and especially a good microphone is important. Like headphones, having a good mic just makes life nicer for others on the call.

A good mic doesn’t need to be expensive, or even a special purchase. Most things will be better than the built-in mic on your laptop. If you have to take notes, the built-in mic will make the typing noises will be a huge booming nuisance. “Could whoever is doing the angry typing please stop?” was the most common phrase on my 2019 calls (in 2020, in desperation, we finally worked out the ‘mute all’ button, leading to “you’re on mute” taking the number one spot).

Airpods seem to have good sound pick-up, if you can tolerate in-ear buds for long periods. Be aware, though, some other bluetooth headsets are pretty awful. It took me a while to work out why no one could understand me on one of mine.

Test, test, test

Most of us wouldn’t dream of writing code without testing it, but we’ll cheerfully broadcast ourselves without testing first. It’s easy to do some quick experiments to see what works best. If you’re using a Mac, use Quicktime to record yourself, either audio-only or full video. Experiment with a few different positions, see what happens if you get excited or hushed, and try typing. Then watch it back. (I know, it feels awful, but it’s the right thing to do.)

Once you’re happy, double-check by doing a recording on zoom or webex or whatever you use most often, and then (sorry!) watch that back too. Your colleagues will thank you.

Programming in the pandemic: personal insights & professional process - CW Developer Network

Wed, 03 Feb 2021 00:00:00 GMT

The Computer Weekly Developer Network examines the impact of Covid-19 on the software application development community. With only a proportion of developers classified as key workers (where their …

Remote working: 10 lessons leaders should apply in 2021

Mon, 11 Jan 2021 00:00:00 GMT

In 2019, the Enterprisers Project made a number of end-of-year predictions: One of them was that remote work would increase in 2020. It did, though none of us wanted or imagined a pandemic prompting the near-overnight remote shift. I haven’t been in the office since March, even though I love my office.

No, Really, Cloud Native Is About Culture, Not Containers

Sun, 10 Jan 2021 00:00:00 GMT

For most of 2019 and 2020, I felt uneasy and slightly guilty about the Cloud Native Computing Foundation’s definition of ‘Cloud Native’. The CNCF definition had a strong implication that you weren’t Cloud Native if you weren’t using microservices.

Want to Save the World? Start by Cutting Your Cloud Costs

Wed, 06 Jan 2021 00:00:00 GMT

How the IT sector can minimize the damage it does to the earth’s climate.

97 Things Every Cloud Engineer Should Know

Thu, 31 Dec 2020 00:00:00 GMT

If you create, manage, operate, or configure systems running in the cloud, you’re a cloud engineer—even if you work as a system administrator, software developer, data scientist, or site reliability … - Selection from 97 Things Every Cloud Engineer Should Know [Book]

WTF is Cloud Native Culture (Container Solutions WTFinar)

Tue, 15 Dec 2020 00:00:00 GMT

It’s really easy to spin up a project in the cloud these days. So easy, millions of abandoned apps are actually contributing to global warming.

The Cloud Should Be Fun - and If It’s Not You’re Probably Doing It Wrong

Tue, 24 Nov 2020 00:00:00 GMT

Is the cloud a fun place? Does the question even make sense? Is fun at work possible? As a consultant with the IBM Garage, Holly has seen fun in the cloud and … un-fun in the cloud. In this talk, she’ll share how to use fun as a guide to getting the most out of the cloud

How to Love Kubernetes and Not Wreck The Planet Part I: Elasticity and Utilization

Fri, 06 Nov 2020 00:00:00 GMT

I gave a keynote at KubeCon Europe. Our community should be thinking about climate change and how kubernetes impacts the climate when it is used inefficiently.

How to Love Kubernetes and Not Wreck The Planet Part II: Revenge of the Zombies

Fri, 06 Nov 2020 00:00:00 GMT

What are the climate impacts of our technology habits? Part I of this blog discusses the importance of multi-tenancy, utilisation, and elasticity. So what happens if we establish good multi-tenancy…

Stack Overflow Podcast 283: Cleaning up the cloud to help fight climate change

Tue, 03 Nov 2020 00:00:00 GMT

It’s really easy to spin up a project in the cloud these days. So easy, millions of abandoned apps are actually contributing to global warming.

Cloud Chaos and Microservices Mayhem

Fri, 23 Oct 2020 00:00:00 GMT

The way we use Java has changed. Applets - or any browser-side Java - are a long, long, time ago. Java moved to the back end, and stayed there, but the back-end has gotten bigger, more distributed, and more complicated. Some things that were a good idea ten years ago turn out to be a terrible idea in the cloud; and what used to be ‘good enough’ for testing really isn’t anymore. Managing microservices architecture demands a lot of us, to ensure observability, operational resiliency, and organisational agility. This talk will introduce some of the new tools, patterns, and best practices for Java development in 2020. It also gives a tour of some of the most painful anti-patterns Holly has seen as a consultant with the IBM Garage.

How - and why - to modernize your scruffy old Java applications

Thu, 15 Oct 2020 00:00:00 GMT

Once upon a time, that application was great - it was built using the latest technology, it was easy to understand, users loved it. Now, not so much. It’s grown and looking a bit … unwieldy monolithic. We’ll discuss when to modernise, when to leave well enough alone, and what you need to know before you microservice. Automation and testing are important parts of a modern app, so we will demo how the Pact contract testing framework can save effort, improve flow, & protect against regressions.

KubeCon + CloudNativeCon Europe 2020 Keynote Summary

Sun, 30 Aug 2020 00:00:00 GMT

The annual KubeCon + CloudNativeCon Europe event ran as a virtual conference in August. Key takeaways included: there is an increasing importance in understanding end-user adoption patterns of technol

People, Planet, Clouds

Thu, 09 Jul 2020 00:00:00 GMT

Using Lease Resources to Manage Concurrency in Tekton Builds

Thu, 09 Jul 2020 00:00:00 GMT

I’ve recently started using Tekton as my main build system. In combination with Argo CD for GitOps management of the pipeline scripts, it has some pretty nice features. One of the things I’m…

How to Become an IBM Garage Developer

Fri, 26 Jun 2020 00:00:00 GMT

The IBM Garage is a highly dynamic, multi-disciplinary consulting practice working with clients and partners of any size across all industries. We combine design thinking, lean startup, devops…

The Climate Service + IBM Garage: Weathering Disruption to Deliver Results

Mon, 04 May 2020 00:00:00 GMT

97 Things Every Java Programmer Should Know

Fri, 01 May 2020 00:00:00 GMT

If you want to push your Java skills to the next level, this book provides expert advice from Java leaders and practitioners. You’ll be encouraged to look at problems in

Cloud Native is about Culture, not Containers

Mon, 02 Mar 2020 00:00:00 GMT

As a developer in IBM’s Cloud Garage, Holly Cummins works with customers who are trying to shift their businesses to the cloud, and to cloud native in particular. Their dream is more effort higher up the value chain, more innovation, and greater adaptability. What they really want is to beat their competitors to market, with something that’s better than their competitors, and then evolve it to beat any new competitors. What’s getting in their way isn’t the technology—wrapping something in a docker container (usually) isn’t that hard. Instead, it’s the structures that have been put in place to manage risk and the relationships between teams that trip companies up. Holly shares stories of customers struggling to get cloud native and explains how IBM applied its methodology to turn things around. You’ll learn the ideal team size, the ideal microservice size, what skills a team needs, the role of architects, how to know if something is ready to ship, and whose fault everything is (joke).

Nine Ways to Fail At Cloud Native

Wed, 26 Feb 2020 00:00:00 GMT

Cloud native is the perfect recipe for innovation, adaptability and engineering excellence – when it goes right. When it’s not right, it can be a monster spaghetti, a quality headache, and frustratingly inflexible. Why so negative? The physicist Niels Bohr said “An expert is a person who has found out by their own painful experience all the mistakes that one can make in a very narrow field.” In the IBM Garage, I co-create cloud native applications with clients. We end up in a good place, but sometimes there are a few bumps on the way. 1 – The Magic Morphing Meaning What even is cloud native? Built for the cloud? Running on Kubernetes? Microservices? Modern? Idempotent? If you ask five people to define cloud native, you might get seven diffe

Garbage Collection Is Your Friend

Mon, 03 Feb 2020 00:00:00 GMT

Poor old garbage collection. One of the unsung heroes of Java, often blamed, rarely praised. Before Java made garbage collection mainstream, programmers had little choice but to track all the memory…

仕事場における楽しさの重要性

Mon, 03 Feb 2020 00:00:00 GMT

仕事において笑顔になることや、笑ってしまうことはチームの団結、生産性、組織のパフォーマンスを証明してくれる。楽しさは強制できるものではないが、促進できるものであると Holly Cummins 氏は FlowCon France 2019 で語った。仕事場における楽しさの重要性の講演である。

The Importance of Fun in the Workplace (Interview)

Thu, 19 Dec 2019 00:00:00 GMT

Things at work that make us smile or laugh can improve team cohesion, productivity and organisational performance. Fun can’t be forced, but it can be fostered, said Holly Cummins at FlowCon France 201

People, Planet, Clouds

Sat, 23 Nov 2019 00:00:00 GMT

The world is changing. The cloud gives us dazzling computational possibilities, and … potentially uses a lot of energy. As climate change accelerates, where do we, as engineers, fit in? Are we part of the problem or part of the solution? How do we balance the needs of people against the need of the planet? Or can they be aligned?

The Importance of Fun in the Workplace

Wed, 13 Nov 2019 00:00:00 GMT

An in-depth look at what makes software development a roller coaster where the highs of 0 compiler warnings are quickly cancelled out by the pain of long hours, bad requirements, endless configuration, clueless managers and a plethora of other issues which make death by a thousand cuts seem like a good idea…. They will answer questions such as: “Why is programming often called an art despite having its underpinnings in formal logic?” “How can I rediscover the delight I felt when I first started coding?” “What’s that rush I feel when my test passes? Am I addicted to TDD?” Combining Psychology, Philosophy and Computer Science, Dr Holly Cummins and Martijn Verburg will present a series of practical tips to help you rediscover the euphoria that you felt the very first time a metal box in front of you came to life and cried out “Hello World”.

Java Should Feel Fun

Wed, 21 Aug 2019 00:00:00 GMT

I started my Java career using J2EE 1.2. I had questions: Why were there four classes and hundreds of lines of generated code for each bean? Why did compiling tiny projects take half an hour? It…

The Future of Work is (More) Fun - Part 1 - IBM UK Blog

Thu, 20 Jun 2019 00:00:00 GMT

The evidence that fun boosts productivity is compelling. A happy work environment means fewer sick days, harder work, and greater productivity.

The Future of Work is (More) Fun - Part 2 - IBM UK Blog

Thu, 20 Jun 2019 00:00:00 GMT

Automation eliminates toil When the printing press was invented, it almost immediately eliminated the arduous practice of copying books by hand. (Before the printing press, writing out the pages for a single bible would take around three months.) This life-changing mechanization accelerated with the industrial revolution, and many labour-intensive tasks were automated. This included factory activities, such as spinning and weaving, but also domestic activities such as laundry. (Washing a household’s clothes by hand is hard, heavy, soggy, work, and not fun.) As we progressed from the industrial revolution to the computer one, some mental tasks were also automated; I learned when I watched Hidden Figures that in 1950 ‘a computer’ was a person

The importance of fun in the workplace

Tue, 11 Jun 2019 00:00:00 GMT

A shorter version of https://www.slideshare.net/HollyCummins/software-developers-guide-to-fun-in-the-workplace-euphoria-despite-the-despair An in-depth look at what makes software development a roller coaster where the highs of 0 compiler warnings are quickly cancelled out by the pain of long hours, bad requirements, endless configuration, clueless managers and a plethora of other issues which make death by a thousand cuts seem like a good idea…. They will answer questions such as: “Why is programming often called an art despite having its underpinnings in formal logic?” “How can I rediscover the delight I felt when I first started coding?” “What’s that rush I feel when my test passes? Am I addicted to TDD?” Combining Psychology, Philosophy and Computer Science, Dr Holly Cummins will present a series of practical tips to help you rediscover the euphoria that you felt the very first time a metal box in front of you came to life and cried out “Hello World”.

The Importance of Fun in the Workplace

Tue, 14 May 2019 00:00:00 GMT

The Importance of Fun in the Workplace

Wed, 01 May 2019 00:00:00 GMT

The Importance of Fun in the Workplace

Sat, 13 Apr 2019 00:00:00 GMT

The Hero's Journey to Cloud: Why Star Wars, Prometheus, and Cloud Are All Interconnected

Thu, 21 Feb 2019 00:00:00 GMT

In the Cloud Garage, we see some patterns over and over again—an organisation wants to achieve an improvement and realises moving to cloud is the way.

The story of designsparkmarketplace.com

Tue, 12 Feb 2019 00:00:00 GMT

The IBM Cloud Garage worked with RS Components to create designsparkmarketplace.com, a peer to peer maker marketplace. This is the story of how we did it.

Cloud Native is about Culture, not Containers

Mon, 19 Nov 2018 00:00:00 GMT

The Importance of Fun in the Workplace

Mon, 12 Nov 2018 00:00:00 GMT

Cloud Native is about Culture, not Containers

Mon, 12 Nov 2018 00:00:00 GMT

Cloud Native is about Culture, not Containers

Mon, 29 Oct 2018 00:00:00 GMT

What Zebrafish Can Teach Us About Healthy Workspaces

Wed, 10 Oct 2018 00:00:00 GMT

A core part of the IBM Cloud Garage practices is a “sustainable pace.”

Cloud Native is about Culture, not Containers

Wed, 26 Sep 2018 00:00:00 GMT

Gatos, Qubits e Teletransporte: O estranho mundo dos algoritmos quânticos (Parte 3)

Thu, 13 Sep 2018 00:00:00 GMT

Os mesmos fatores que tornam a teoria quântica tão surpreendente também tornam os computadores quânticos em algo muito difícil para ser implementado na prática. Este artigo explora estes desafios.

Gatos, Qubits e Teletransporte: O estranho mundo da computação quântica (Parte 1)

Thu, 06 Sep 2018 00:00:00 GMT

Há vinte anos, computadores quânticos eram puramente teóricos e existiam apenas em quadros brancos e em trabalhos acadêmicos. Agora, os computadores quânticos estão disponíveis para locação na nuvem.

猫、量子位和远距传动：令人匪夷所思的量子计算世界（第三部分）_语言

Wed, 25 Jul 2018 00:00:00 GMT

猫、量子位和远距传动：令人匪夷所思的量子计算世界（第三部分）关键要点让量子理论显得如此惊人的因素同样也让得量子计算机在实践中很难实现：量子现象并未在日常生活中表

猫、量子位和远距传动：令人匪夷所思的量子计算世界（第二部分）_语言

Tue, 17 Jul 2018 00:00:00 GMT

猫、量子位和远距传动：令人匪夷所思的量子计算世界（第二部分）关键要点远距传动是最先被描述的量子信息处理过程之一。它使用纠缠在瞬间远距离传输信息，但它不能用于传送

Cats, Qubits, and Teleportation: The Spooky World of Quantum Computation Applications (Part 3)

Sat, 14 Jul 2018 00:00:00 GMT

The same factors which make quantum theory so startling also make quantum computers very difficult to implement in practice. This article explores these challenges

猫、量子位和远距传动：令人匪夷所思的量子计算世界（第一部分）_语言

Sat, 07 Jul 2018 00:00:00 GMT

猫、量子位和远距传动：令人匪夷所思的量子计算世界（第一部分）关键要点二十年前，量子计算机还只处在理论阶段，只存在于白板和学术论文中。而现在，量子计算机已经在云端

Cats, Qubits, and Teleportation: The Spooky World of Quantum Algorithms (Part 2)

Fri, 06 Jul 2018 00:00:00 GMT

The relationship between complexity theory, quantum algorithms and machine learning is becoming increasingly important. This article explores quantum algorithms and their applicability.

Cats, Qubits, and Teleportation: The Spooky World of Quantum Computation (Part 1)

Fri, 29 Jun 2018 00:00:00 GMT

Twenty years ago, quantum computers were purely theoretical, and existed only on whiteboards and in academic papers. Now, quantum computers are available for hire on the cloud.

Cloudy with a Chance of Meatballs: Cloud Surprises for the Java Developer

Tue, 13 Mar 2018 00:00:00 GMT

Many businesses are moving to the cloud. This journey to the cloud is in fact a quest, complete with a hero (us!), a call to action, a perilous journey, and a glittering reward (no more patching operating systems!).

The importance of fun in the workplace

Tue, 13 Feb 2018 00:00:00 GMT

One of the key elements of the Cloud Garage Method is Culture. That single word encompasses a whole range of practices about how we manage our projects, and how we interact with our stakeholders and each other. One of my favorite parts of the Cloud Garage culture is the fact that it includes a fun work environment. Although we all like fun (I hope), it’s often seen as slightly unprofessional – fun is something that happens outside work. Most of us would be a bit nervous about writing, “I’m great fun” on our CVs, or reporting to our boss how much fun we’re having in the office. Nonetheless, fun matters a lot, and not just because it’s nice. Why fun matters The evidence that fun boosts productivity is compelling. A happy work environment mean

Working out how much a project will cost

Tue, 06 Feb 2018 00:00:00 GMT

When customers visit the IBM Cloud Garage to discuss a potential project, one of the most frequently asked questions is “how much is my project going to cost?”. It’s generally accepted that – without a working time machine – at least one corner of the project management triangle needs to be flexible. Trying to nail down scope, cost, and schedule at the beginning of a project is a notorious anti-pattern; delivering a pre-determined list of features on-schedule within a given budget may give the illusion of success, but quality will almost always suffer in some way. These latent problems, or technical debt, can significantly affect the success of a project in the field, and also the cost of future extensions. In the Garage, we’re convinced th

Chatbot Best Practices

Mon, 29 Jan 2018 00:00:00 GMT

One of the most frequent questions clients ask when visiting a Cloud Garage is “Can you build us a chatbot?” This question is reflective of an industry-wide trend towards more natural language in computerised interactions, and also more automation of interactions currently handled by humans. Today, there are currently more than 33,000 chatbots on Facebook Messenger alone. Many businesses are turning to Watson Conversation to help take out cost and improve user satisfaction. Our Hursley Labs colleague Simon Burns has written an excellent series of articles on how to write great Watson chatbots, which you should definitely go read. Think of this blog as a supplement, with our experiences from the field. To address this pressing question, I’ve

Software Developers Guide to Fun in the Workplace: Euphoria Despite the Despair

Wed, 10 May 2017 00:00:00 GMT

Innovation Stories from the Bluemix Garage

Wed, 05 Apr 2017 00:00:00 GMT

Everyone’s talking about innovation, but how do you know if you’re actually doing it? What are the ingredients for successful innovation? In this talk, Holly will describe how the right combination of people, place, practices, and platform can lead to some pretty impressive outcomes. She’ll also answer questions, such as ‘what happens when we think about our user first?’, ‘is there an app for that?’, ’can a computer really tell dog breeds apart?,’ how can I tell if my idea is great or terrible?’, ’how long does it take to build a minimum viable product?’

Thinking Strategically About IoT

Mon, 06 Mar 2017 00:00:00 GMT

Cool? Useful? Disruptor? All of the above? IoT is having an impact on more and more industries. As the cost of instrumenting things and collecting data drops, the possibilities for what we can control and the kind of insights we can gather increase. Not only is IoT hardware cheaper and more pervasive, developing IoT software is now far more accessible. That doesn’t mean there aren’t tricky bits. Does Java have relevance in the IoT world? How can you keep the system reliable and handle failure in a cost-effective way? How can you cope with the data volumes? What’s the best way to turn raw data into insight?

Thomson creates travel search chatbot with IBM Watson

Tue, 22 Nov 2016 00:00:00 GMT

Travel search tool uses Watson’s natural language processing and speech-to-text capabilities.

IBM Garage radically changes how you deliver tech-enabled innovation

Mon, 14 Nov 2016 00:00:00 GMT

How well do you know the Bluemix Garage? CTO Rachel Reinitz calls it “a collaborative environment to work as one team with clients to rapidly define, design, develop, validate, analyze, and deploy successful apps.” In fact, she says, “Our Bluemix Garage Method melds industry best practices, startup culture, and IBM’s expertise, and adapts these for the needs of our clients, large or small.” But that’s only the beginning. How do most Bluemix Garage projects get started? Most innovation projects start with a two-day design thinking workshop. Before starting the workshop, businesses have identified an area of opportunity (but not a solution—that’s what the workshop is for). Identifying the right mix of participants is also important. Diverse p

Microservices: from dream to reality in an hour

Sun, 18 Sep 2016 00:00:00 GMT

Are microservices a wonder-pattern for rescuing intractably complex applications? Or are they just a restatement of the software engineering best practices we all should be following anyway? Or something in between? How do they work? How should they be written? What are the pitfalls? What are the underpinning technologies?

Building Stuff for Fun and Profit - confessions from a life in code and cables

Fri, 08 Jul 2016 00:00:00 GMT

I love making stuff. I’m so happy that my job allows me to make stuff, and when I’m not at work, I’m making stuff anyway. Some of the stuff I’ve made has solved real technical and business problems; some of it I’ve done just to see if I can. In this talk I’ll describe some of the valuable things I’ve built for my employer, IBM, and our clients - I’ll also describe some of the ridiculous things I’ve made for myself. These are slides for a talk given at BuildStuff Odesa, 2016.

Putting the 'Micro' Into Microservices With Raspberry Pi - DZone IoT

Tue, 14 Jun 2016 00:00:00 GMT

How the Bluemix Garage is helping a blind athlete run marathons - solo

Thu, 12 May 2016 00:00:00 GMT

The Bluemix Garage London has written an application to guide a blind-runner on an ultra-marathon in the desert, using a GPS route and beeps.

This blind man is running a 155 mile ultra-marathon with the help of an IBM app

Tue, 03 May 2016 00:00:00 GMT

Simon Wheatcroft is completely blind. But that’s not going to stop him running a 250 kilometer (155.3 mile) race in the Namibian desert.

That’s not useless, it’s just … unusual

Thu, 24 Mar 2016 00:00:00 GMT

Talk presented at the London Java Community’s Mad Scientists event: http://mad-scientists.mybluemix.net. Innovation is the new currency of business, but let’s not forget the sheer silly joy of creativity. I’ll be the first to admit that the business value of world’s first cuddly throwable application server is pretty low. The application server embedded in a hat is probably even less lucrative. The good news is that I’m part of a long tradition of inventing useless things - and we’re still benefiting today from some of those apparently useless things. So even if the throwable application server never goes to market, lets all keep trying things out just for the wonder of it. Even if everyone else thinks we’re bonkers, we’ll be having fun.

Microservices: from dream to reality in forty five minutes

Mon, 08 Feb 2016 00:00:00 GMT

Microservices: from dream to reality in an hour

Mon, 12 Oct 2015 00:00:00 GMT

Confessions of an automation addict

Fri, 18 Sep 2015 00:00:00 GMT

I’ve never seen a job I didn’t want to automate. Sometimes it’s worked out well, sometimes automation has turned a small nuisance into a big, fragile, free-time-eating monster nuisance. In this talk, I’ll explore why we automate, when to automate, the hazards of automation and the - big - rewards of automation. I’m part of the team developing IBM’s WebSphere Liberty application server. We’ve used a mix of off-the-shelf and home-rolled tools and processes to work smarter and more productively. I’ll describe what we’ve learned as WebSphere has transitioned to DevOps and continuous delivery and why I still can’t resist trying to automate all the things.

Microservices: Storm in a teacup, or teacups in a storm? - JAXenter

Mon, 07 Sep 2015 00:00:00 GMT

Somehow, the buzz around microservices has us believing that every single employee and enterprise must break up their monolith empires and follow the trend.

The Cuddly Throwable Application Server

Wed, 04 Mar 2015 00:00:00 GMT

Computers are getting small enough and cheap enough that they’re almost disposable. It’s possible to sock computers away almost anywhere, and to connect almost anything to the internet. At the same time, the Java stacks that we know and love are also getting lighter and cheaper. That combination means it’s possible to put a full spec-compliant Java EE server on ridiculously cheap hardware, and then throw it around the room without worrying too much about breaking stuff. Source code: https://github.com/holly-cummins/throwable-application-server

Interacting with headless computers (or "How to not keep losing your raspberry pi on the network")

Wed, 25 Feb 2015 00:00:00 GMT

I spend a fair amount of time playing with Raspberry Pis and other ARM-based boxes, like pcDuinos and Utilite. (If my boss is reading this, I’m not “playing”, I’m “working”.) I still find it amazing how small, cheap, and powerful these devices are.

One thing it’s taken me a while to work out is how to connect to the computer. It’s kind of a prerequisite for doing anything more interesting, like embedding it in a hat or a ball, or just … using it. It should be so easy, right? A lot of set-up guides start with you plugging the processor to an HDMI monitor and USB keyboard. This isn’t so practical if I’m out and about with the computer, because I’m not going to be lugging an HDMI monitor with me. Even at home, it’s not foolproof - my new pcDuino doesn’t seem able to display a picture over HDMI to any of the (six) monitors, TVs, or projectors I tried. At that point it’s not so much “headless” as “decapitated”.

Connecting by ssh requires less bulky hardware, but it has its own challenges: what on earth is the ip address of this thing? I used to ensure I always knew the ip address by nailing it down. I’d connect the pi to my laptop with an ethernet cable (no need for a fancy crossover one with a modern computer), configure the pi with a static ip address (for example, 192.168.2.10), and then configure my laptop’s ethernet interface to think it was 192.168.2.1. This ensured I could easily connect to my pi, but it did require some initial set up of the network. For my totally inaccessible pcDuino, this is no good. Even for the pi, it’s not ideal - the static ip address means then even if I plug it into my router, it’s totally divorced from the “real” internet. More seriously, I’ve had a few panics where I’ve loaned the pi out and then found I was unable to connect to it when I got it back. (Curse you, nameless colleague who re-configured the pi to use dhcp and didn’t tell me!) The static ip settings also left my laptop in a pretty messed up state.

I’ve now found a new system which avoids some of these disadvantages. I don’t touch the ethernet configuration of the arm box, so it can stay using dhcp. I connect it to my laptop with an ethernet cable but I configure my mac to share its internet connection from wi-fi to ethernet. This OS magic enables the two computers to see each other without either having to have crazy network settings.

I then use nmap to find the pi on the network, as follows:sudo nmap -sn 192.168.1.0/24. To work out which ip address to scan, I run ifconfig and use the ethernet access of the bridge100 network interface (but change the last digit to 0). The sudo isn’t necessary, but it makes nmap show the MAC addresses.

This isn’t so useful for the pcDuino, which doesn’t have a fixed MAC address, but it really helps identify the pi, which will always have a MAC address in the b8:27:eb range.

Edit - other options: I haven’t tried these, but twitter has spoken! If you have a flock (or even a small handful) of identical devices, including ipspeak in the bootable image bypasses the need for any mmap sleuthing. You can plug headphones into the device and it will read out its ip address every thirty seconds. Alternatively, if you have individually configured devices, avahi allows them to be addressed using a memorable name.

Acknowledgements

There are many similar guides on the internet. I’ve posted this here mostly in the hope it will be useful notes to my future self, since I’ve independently rediscovered my optimum flow about four times. I figured out the nmap commands from a relatively recent post at stackoverflow, and the idea of using network sharing to connect my pi to my laptop without wrecking my laptop’s settings is from alvin_jin.

The Wearable Application Server and Other Adventures in Software Engineering

Mon, 03 Nov 2014 00:00:00 GMT

The Lazy Developer’s Guide to Cloud Foundry

Thu, 12 Jun 2014 00:00:00 GMT

Enterprise OSGi In Action

Fri, 01 Mar 2013 00:00:00 GMT

Enterprise OSGI in Action is a hands-on guide for developers using OSGi to build the next generation of enterprise Java applications. By presenting relevant examples and case studies, this book guides the reader through the maze of new standards and projects.

OSGi vs Spaghetti - Part II, The Enterprise strikes back

Mon, 04 Feb 2013 00:00:00 GMT

Software engineering sometimes seems like a struggle against spaghetti. Without lots of care, modern projects risk becoming an unwieldy tangle of closely-coupled code whose behaviour and dependencies are mostly unknowable. OSGi can help keep the complexity at bay, but until recently OSGi didn’t play very well with modern enterprise programming models such as dependency injection, WARs, JPA, and application deployment. The Enterprise OSGi programming model is changing all this, and can help keep your applications from descending into a noodly mess. This session will discuss the basics of OSGi modularity, and demo the build tools, middleware, and architectural principles which can be used to turn a chunky WAR into a lean, mean, extensible OSGi web application.

The Wearable Application Server and Other Adventures in Software Engineering

Mon, 04 Feb 2013 00:00:00 GMT

Mobile technology has so far mostly been confined to the client side, for fairly obvious reasons - traditionally, clients are mobile, and servers are not. However, not only is hardware getting smaller, servers are too. When your application server can run on pocket-sized £25 hardware it opens up some pretty cool possibilities - your server is literally lightweight. Not only can you have location-based services, you can have locatable servers. Servers can run on phones, they can run on the Raspberry Pi, and so they can go almost anywhere you can think of. Modularity gives software the flexibility it needs to cram into these tight spaces without sacrificing power. This talk will demonstrate developing and deploying a web application to an application server embedded in a silly hat.

Building Modular Applications with Enterprise OSGi

Mon, 12 Nov 2012 00:00:00 GMT

Everything I know about software, I learnt from spaghetti bolognese

Mon, 15 Oct 2012 00:00:00 GMT

IBM’s Holly Cummins and ‘Diabolical Developer’ Martijn Verburg discuss ways of overcoming complexity in software development. None of of us enjoy spaghetti code, much less spaghetti builds, spaghetti tests, or spaghetti classpaths. Some days software engineering seems like the struggle against spaghetti - modern projects often become a large and complex tangle of source code and resources, and the runtime interactions are even worse! This means many projects are hard to navigate, difficult to extend and problematic to deploy/run. You only have to look at the OpenJDK itself to see the problems it faces in trying to have a smaller deployment and runtime footprint! Jigsaw was to be the inbuilt solution for the OpenJDK, extending beyond the JVM itself to help applications with their modularisation story (i.e. Jigsaw or OSGi). Sadly, Jigsaw is not going to make it for Java 8, so developers are left with only a few tooling choices in order to help them. However, tooling is only part of the answer in managing the complexity. What really matters in avoiding a noodly mess is the layout of your source code and resources, clear contracts, and how you think about the dependencies your code has at runtime. Dr Holly Cummins and Martijn Verburg will take you through some of the practical design decisions you can take to help modularise your application properly, independent of tooling. Time permitting they’ll then showcase some of tooling support that OSGi can bring to the table as the premier modularity system for Java today.

Java basics: converting a collection to a string

Thu, 27 Sep 2012 00:00:00 GMT

If you look up how to write a map or a list out as a string, you can find lots of complicated answers involving loops and even XMLEncoders. It doesn’t have to be that hard. The easiest way to convert a Map into a nice string?

String niceString = Arrays.toString(map.entrySet().toArray());

It’s even easier for a list:

String niceString = Arrays.toString(list.toArray());

Ta-daaa!

OSGi and the Enterprise - A match made in a … box?

Tue, 08 Jun 2010 00:00:00 GMT

The Enterprise OSGi specification provides a compelling programming model for creating modular, portable, and dynamic enterprise applications. The Apache Aries project is an open source implementation of this standard. It is designed to complement existing web servers with a new set of pluggable enterprise-level services such as transactions, persistence, service bindings, and dependency injection. The Blueprint container standardizes the popular Spring component model, enabling dependency injection, application component assembly, and allowing beans to be easily swapped in and out for unit testing. Integration with JPA and JNDI standards allow declarative persistence and interoperability with legacy applications. Support for transactions and JMX provide enterprise-level application control.

Java performance - not so scary after all

Wed, 12 May 2010 00:00:00 GMT

No one likes slow applications, but sometimes it’s hard to know where to start when trying to fix a performance problem. This talk will cover a range of tools and techniques which can be used to track down and fix performance issues. Topics covered: Why performance really really matters What’s the garbage collector doing? (And why you should care.) But why is the garbage collector doing all that, anyway? How to find out what’s in your heap. Are you waiting around on locks? Is your application running the code it should be? Pulling it all together

Health Center 1.0 released

Tue, 16 Jun 2009 00:00:00 GMT

Last week was a big week for the Health Center team. Version 1.0 of the Health Center was released. Full installation instructions are available from the Health Center homepage. It now works with IBM Support Assistant and is suitable for use in production (with recent IBM JVMs). A method profiler that’s got a low enough overhead to be used in production and left on all the time is a pretty special thing, I think. One which is only part of a tool which also assesses system stability, triages a range of performance problems, and makes performance tuning accessible to non-experts is even more special, I think.

To celebrate the whole team baked and brought in carrot cake, brownies, chocolate chip cookies, lemon cake, rocky road squares, and flapjacks. We even had balloons. It was very nice but I’ve never eaten so much at work and I think we were all ready to explode by the afternoon - in future I think we should delegate and have only half the team bake per release.

How to interpret a method profile

Tue, 12 May 2009 00:00:00 GMT

In a previous post, I described the general methodology I use to diagnose performance problems.

Once an application has been identified as CPU-bound, either by using the Health Center or CPU monitoring, the next step is to figure out what is eating CPU. In a Java application, this will usually be Java code, but it could be native code. Profiling native code usually requires platform specific tools; on linux, I use tprof. Profiling Java code is a lot easier, and is more likely to yield big performance improvements, so I usually start with a Java profile and only profile native code if I didn’t get anywhere with the Java profile. For Java profiling, I use the Health Center. It’s got a few advantages, one of which is that there’s no bytecode instrumentation needed, there’s no need to specify only a few packages to profile, and the overhead is very low, so it won’t affect the performance characteristics of what you’re trying to profile.

So what does a method profile tell you? Simply put, it tells you what your application is spending its time doing. More precisely, it tells you what code your application is spending its time running - it doesn’t tell you when your application is waiting on a lock instead of running your code, and it doesn’t tell you when the JVM is collecting garbage instead of running your code. Assuming locking and GC aren’t the cause of the performance problem (see triaging a performance problem), the method profile will give you the information you need to make your application go faster.

The application is doing too much work, and that’s slowing it down. Your aim in performance tuning is to make the application do less work. There are lots of ways to make code more efficient. Sometimes people start performance tuning by code inspection - they read through the code base looking for obvious inefficiencies. I’ve done this myself lots of times, but it’s not a particularly efficient technique. Say I find a method which is pretty carelessly implemented, and I double its speed with a bit of refactoring. Then I triumphantly re-run my application, only to discover nothing’s changed. What’s going on? The problem is that a big performance improvement on a method which is rarely called isn’t going to change much of anything. For example, if I double the speed of a method which uses 0.5% of my CPU time, I’ve sped my application up by an imperceptible 0.25%. If, on the other hand, I shave 10% of the time of a method which is using 20% of my CPU, my application will go 2% faster. So the first rule of performance tuning is to optimise the methods at the top of the profile and ignore the ones near the bottom.

This is example from a method profiler, in this case the one in the Health Center. One method is clearly using more CPU than the rest, and so it’s coloured red. In this case, 60% of the time the JVM checked what the application was doing, it was executing the FireworkParticle.animate() method. This is what’s shown by the left-hand ‘Self’ column. The ‘Tree’ column on the right shows how much time the application spent in both the animate() method and its descendants. Some profilers call this column ‘descendants’ instead. Usually the Self figures are more useful for optimising an application.

What makes a method appear near the top of a method profile? It’s taking up a lot of CPU time, but why? There are two reasons; either the method is being called too often, or the method is doing too much work when it’s called. Sometimes it happens that a method really is doing the right amount of work the right number of times, but this is usually only the case after a fair amount of work. In their natural state, most programs can - and should - contain inefficiencies. (Remember that premature optimisation is the root of all evil.)

Some profilers can distinguish between a method which is called several times, and one which is called once and then spends a long time executing, but many cannot. The reason is that some profilers operate by tracing - that is, recording every entry and exit of a method. This gives very precise information, but usually carries a fairly heavy performance cost. The IBM JVM can be configured with launch parameters to count or time method executions, but it’s only advisable to do this for a restricted subset of methods. An alternate method of collecting profiling information is to sample - that is, check periodically what method is executing. This is much less expensive but doesn’t give as much detail as tracing profilers. The Health Center uses method sampling already built into the JVM to allow profiling with extremely low overhead.

Often it will be obvious when inspecting a hot method if it’s being called frequently or is slow to run. Code with loops, particularly nested loops, is probably expensive to run. Code which doesn’t seem to do much but which is at the top of a profile is probably being called a lot. This leads neatly to the next steps in optimization: eliminate loops and do less work inside loops for expensive methods, and call inexpensive method less frequently.

How do you go about making sure a method is called less? Method profilers which also record stack traces can make calling method less pretty easy. For example, this is the output of the Health Center, showing where calls to one of the top methods in the profile have come from:

In this case, 98% of the time the doSomeWork() method was sampled, it was animate() that called it. 2% of the time, it was draw() that called it. In this case, the next step is to inspect the animate() method and see why it’s calling doSomeWork(). Often, at least in the first passes of optimisation, most of the calls to the top method are totally unnecessary and can be trivially eliminated.

How do you solve a performance problem?

Mon, 11 May 2009 00:00:00 GMT

I was talking to a friend today about a performance problem he was seeing, and he explained that he wasn’t entirely sure how to fix it. Performance tuning can be very skilled, but the basic principles are straightforward and should be accessible to every Java developer.

This is the methodology I recommend.

All performance problems are caused by a limited resource. Your job as a performance analyst (or developer who’s suddenly required to be a performance analyst) is to identify what resource is limited - what’s the bottleneck for this application? Often after fixing the first bottleneck, a second bottleneck will become apparent - the process of performance tuning is the process of eliminating bottlenecks, one by one, until the performance is good enough for you and your stakeholders.

Computational resources fall into a few basic categories. Different people count them differently, but I like to think of four types of resource: the CPU, memory, I/O, and locks.

A CPU-bound application can’t get enough processor time to complete its work. A memory-bound application needs more memory than is available. An I/O-bound application is trying to do I/O faster than the system can handle. Finally, a lock-bound application is being held up by the fact that multiple threads are contending on the same locks. ‘Lock-bound’ isn’t a terribly common term, but I think it’s really important to consider lock contention when analyzing performance problems. As systems become more and more parallel, synchronization are increasingly a limiting factor in the scalability of the system.

So how do you identify which of these resources is the cause of the hold-up? Some heuristics can help as a first step. If the CPU is at or near 100%, the CPU is likely to be the culprit. If the CPU isn’t near 100%, it’s probably locking or I/O. The rule of thumb becomes a bit muddier when it comes to memory. Sometimes memory problems can show up as low CPU usage, because the system is waiting on a heap lock (which is properly locking) or physical memory access - paging in the worst case (technically, I/O). However, in a garbage collected system, excessive memory consumption will often manifest as lots of processing time being spent in the garbage collector (CPU).

Tools can help turn these fairly fuzzy heuristics into a more precise diagnosis. There are lots of tools available, both free and not-free. My favourite tool is IBM’s Health Center. This is probably because I’m the technical lead for the Health Center, so I think what it does is pretty sensible and cool. :) The Health Center is free, but it can only monitor IBM JVMs. I do think it’s one of the best ways to investigate garbage collection, locking, and collect method profiles from an IBM JVM.

The Health Center tries to automate the process of identifying the root cause of a performance problem. The front page shows a dashboard with a bunch of status indicators. If one of them is red or orange, that’s a pretty good indicator of where to start tweaking the performance. The one area the Health Center doesn’t cover at the moment is I/O. It can identify garbage collection and locking bottlenecks pretty accurately, and the method profiler can identify the root cause of much excessive CPU usage.

So let’s say you’ve used the Health Center (or your tool of choice) and you’re seeing some red crosses. What next? In the Health Center, clicking on the link next to a red cross will bring up more information and more detailed recommendations about how to fix the problem. In later posts I’ll give a bit more background about how to go about fixing locking issues, memory issues, and CPU issues.

Forward links:

How to interpret a method profile

Garbage collection flavours

Wed, 08 Oct 2008 00:00:00 GMT

In my previous post, I described the basic garbage collection algorithms. Within these basic categories, there are many variants, or flavours.

Generational collectors

Generational collectors exploit the observed properties that most objects tend to die young, and that young objects are more likely to reference old objects than the reverse. Together these are known as the weak generational hypothesis.

Generational collectors divide the heap up into multiple generations. Young generations can be collected without collecting old generations. These partial collections are quicker than full heap collections, and are likely to produce a good return of free space relative to the area collected, since most objects die young. At least the younger generations tend to employ copying collectors, since these collectors are very efficient in heaps with high attrition rates.

Objects which survive a given number of collections are tenured and move up to an older generation. In order to avoid collecting objects in younger generations which are referenced by older generations, a remembered set is maintained of references from old generations to young generations. A write barrier between the generations is used to catch changes to these references.

Incremental collection

Incremental collectors allow single collections to be divided into smaller collections. This allows pause times to be limited. Incremental techniques almost always require one of a write barrier or a read barrier to prevent changes to the object connectivity graph mid-way through a collection causing the reachability results to be incorrect.

Concurrent collection

A concurrent collector is one which can execute concurrently with a mutator thread. Even on moderately multi-processor systems, it usually does not make sense to dedicate an entire processor to running garbage collection. Therefore the typical concurrent collection is perhaps more accurately described as a highly incremental collection; each thread is assigned small units of garbage collection work to do along with its application work, so the garbage collection work is finely interleaved with application work. One way of thinking of this garbage collection work is as a ‘tax’ - threads have to do some garbage collection work in exchange for being able to allocate. As with incremental techniques, concurrent collections need a write- or read-barrier.

Parallel collection

A parallel collector is one which divides the collection work so that multiple collector threads are collecting concurrently. This is counter-productive on single-threaded processors, but increasingly necessary as the number of processors in a system increases. On systems with many processors a single-threaded collector is unable to keep up with the amount of garbage the processors can produce.

Similar techniques are used to achieve incrementality, concurrency, and parallelism, so many collectors which have one of these properties also have some of the others.

Garbage collection algorithms

Tue, 07 Oct 2008 00:00:00 GMT

Garbage collection has been the subject of much academic research . In particular, the volume of new techniques with various claimed properties is high enough that it is tempting to coin the phrase YAGA (Yet Another GC Algorithm). For a pointer to many of these articles, see Richard Jones’ garbage collection bibliography.

Garbage collection algorithms

Despite the large number of variants, most garbage collection algorithms fall into a few simple categories.

Reference counting

Yes, reference counting is a form of garbage collection. It’s sometimes seen as an alternative to garbage collection, and sometimes even as a morally superior alternative to garbage collection, but really, it’s just a type of garbage collection algorithm.

Reference counting garbage collectors track the number of references to each object. Often the count is maintained in and by the object itself. When a new reference to an object is added, the reference count is incremented. When a reference is removed, the count is decremented. When the count reaches zero, the object is destroyed and the memory released. Reference counting fails when objects reference one another in a cycle. In these cases each object will be seen to be referenced and will never be freed, even if nothing references the cyclic structure.

Reference counting is unable to deal with cycles unless augmented with occasional invocations of another form of garbage collection. Reference counting has a number of other disadvantages, mostly to do with performance and bounding of work, and so it is rarely used in modern garbage collection systems. However, smart pointers in C++, while not traditionally considered a form of garbage collection, do make use of reference counting. Ad hoc garbage collectors introduced to manage object reachability in complex environments also tend to use reference counting since it is easily implemented in a hostile environment.

Tracing collectors

Most garbage collectors perform some sort of tracing to determine object liveness. Tracing does not have the same problem with collecting cycles as reference counting does. However, tracing collectors are more sensitive to changes in the object graph and usually require the suspension of the application threads or close monitoring of application activity to ensure correctness.

Mark-sweep

Mark-sweep collectors collect in two phases. In the first phase, all reachable objects are traced, begining from a set of roots. The roots are all objects which are guaranteed reachable, including objects referenced from the stack and static variables. Every object directly or indirectly reachable from a root is marked. Objects which do not end up marked are unreachable and therefore cannot be used again by the application, so they may safely be freed.

In the second phase of a mark-sweep collection, all unmarked objects are added to a free-list. Contiguous areas of free memory are merged on the free list.

Mark-compact collectors

Mark-compact mark live objects using identical techniques to those used by mark-sweep collectors. However, mark-compact collectors do not use a free list. Instead, in the second phase all reachable objects are moved so that they are stored as compactly as possible. New allocation is then performed from the empty area of the heap.

Many collectors hybridise these two approaches, combining frequent sweeping with occasional compaction.

Copying collectors

Copying collectors divide the heap into two areas, a to-space and from-space. All objects are allocated in the to-space. When the new space is full, a collection is performed and the spaces are swapped. All reachable objects in the to-space are copied to the from-space, which is declared the new to-space. New allocation is then performed in the new to-space.

Copying collectors have a number of advantages over marking collectors. Because copying is done in the same phase as tracing, it is not necessary to maintain a space-consuming list of which objects have been marked. Because no sweep is performed, the cost of the collection is proportional only to the amount of live data.

If most objects are unreachable, a copying collecton can be very efficient. The corollary is that if most objects remain reachable, or if some very large objects remain reachable, the collection will be woefully inefficient, because a large amount of memory will need to be copied every collection.

Copying collectors also keep the heap very compact, and this can boost allocation and application performance. However, because a completely empty from-space must be maintained at all times, copying collectors are not space-efficient. Modern collectors tend to estimate an object death-rate and maintain less than half the heap for the from-space accordingly. Copying collectors are also known as semispace collectors.

For a good overview of how this relates to the IBM JVM’s garbage collection, see Mattias Persson’s developerWorks article.

Garbage collection myths

Sun, 05 Oct 2008 00:00:00 GMT

I’ve been lucky enough in my work to learn a fair bit about garbage collection. One of the things I’ve discovered is how many myths and half-truths exist about garbage collection. Part of the reason these myths have sprung up is that garbage collection has some pretty paradoxical properties. The first myth is that garbage collection is only suitable for the incompetent, unskilled, or lazy. In fact garbage collection offers many architectural and software engineering advantages, even to the skilled developer. The second myth is that garbage collection is all about about collecting garbage. It seems obvious from the name, but it’s not true! Garbage collectors also include an allocation component, which, along with their powers of object rearrangement, can make a significant difference to application performance. Thirdly, criticisms of garbage collection often focus on the pause times, and responses to these criticisms often focus exclusively on reducing pause times, in the mistaken belief that small pause times guarantee good application response times. Pause times are also often used as a metric of general application performance, and an increase in pause times is taken as an indicator of worsened performance, when in fact the opposite the opposite is often true. Paradoxically, even the total amount of time spent paused for garbage collection is not a good predictor of the impact of garbage collection on application performance. Finally, the sixth myth is that garbage collection has a disastrous performance impact. While garbage collection can hurt application performance, it can also help application performance to the point where it exceeds the performance with manual memory management. I’ll go through each of these in detail in later posts.

But, to start off with, what is garbage collection? Garbage collection is a system of automatic memory management. Memory which has been dynamically allocated but which is no longer in use is reclaimed for future re-use without intervention by the application. Garbage collection solves the otherwise difficult problem of determining object liveness by freeing memory only when it becomes unreachable.

Garbage collection is pretty ubiquitous in modern languages. Garbage collected languages include Java, the .Net languages, Lisp, Python, Perl, PHP, Ruby, Smalltalk, ML, Self, Modula-3, and Eiffel. Some languages which are not traditionally garbage collected offer garbage collection as a pluggable or configurable extension. For example, collectors are available for C++, and Objective-C was recently extended to allow garbage collection. Understanding the garbage collector is an important part of performance tuning in these languages.

IBM Monitoring and Diagnostic Tools for Java™ - Health Center™ is now available

Wed, 24 Sep 2008 00:00:00 GMT

I’m part of a team in IBM’s Java Technology Centre which develops diagnostic tools, and we’ve just released a brand new tool. It’s called IBM Monitoring and Diagnostic Tools for Java™ - Health Center™ and it’s available as an early access beta download from developerWorks. We’re really excited about the diagnostic insights it can offer. It allows running JVMs to be observed, health-checked, and tuned. At the moment it gives insight into general system health, application activity, and garbage collection activity. There are visualizations and also recommendations, so you don’t have to be an expert to improve the performance of your application.

One of the nice things about the Health Center is that it presents different kinds of information in one place. There’s a status page which shows at a glance what’s going well and what might need attention:

Garbage collection is one of the first and easiest things to check when tuning an application. Sometimes garbage collection can benefit from a bit of tuning, and sometimes looking at what the garbage collection is doing can highlight application problems. Is your application using more memory than seems reasonable? Is it leaking memory? The Health Center will suggest a command line, diagnose when garbage collection is causing performance problems, and point out memory leaks:

Synchronization can be a big performance bottleneck for applications on multi-CPU systems, and it can be difficult to identify hot locks or even assess the impact of locking. The Health Center has always-on lock monitoring which identifies the objects which are most contended and provides guidance about whether synchronization is hurting performance:

Last, but not least, the Health Center includes a nice sampling method profiler. This can be really valuable in diagnosing applications which are using high CPU. Because it’s low overhead, there’s no need to specify in advance which parts of the application should be profiled. It works without recompilation or byte code instrumentation. It shows where the application is spending its time and gives full call stacks for all sampled methods. Because of the call stacks, this kind of profiling isn’t just for performance problems - sometimes it can give surprising functional insight:

The way the Health Center is able to get so much information without much overhead is that it’s closely linked to the JVM, so there are minimum system requirements. The absolute minimum requirement is an IBM JVM, version 5.0 or higher. This will give locking information and some GC information. To get profiling Java 5 SR8 or Java 6 SR1 is required. The fullest GC information is available in Java 6 service refresh 2. IBM JVMs can be downloaded from developerWorks.

Java Performance Tooling

Wed, 18 Jun 2008 00:00:00 GMT

The right tooling is essential part of any Java performance tuning effort. Tuning relies on accurate diagnoses of bottlenecks; tools allow problem areas to be identified quickly. In some cases tools will even suggest solutions.

Java Performance Tooling

Wed, 21 May 2008 00:00:00 GMT

Java Performance Tooling

Tue, 25 Mar 2008 00:00:00 GMT

Interpreting GC Pauses

Sun, 16 Sep 2007 00:00:00 GMT

I’ve just got back from speaking at JavaZone in Oslo. I’ll post links to the presentation videos when they’re available. The feedback on my presentation and on the Extensible Verbose Toolkit gc analysis tooling I demonstrated was really positive, which was nice.

One of the main things I discussed in my talk is that care is required in interpreting garbage collection pauses. Even though pauses stop an application doing work, which is obviously bad, there’s not necessarily any correlation between shorter garbage collection pauses and better application performance. Sometimes shorter pauses give better performance, sometimes longer pauses give better performance. This is pretty uncontroversial if the quality of service requirement is better throughput. It’s a lot more contentious if response times are the critical performance metric. After my talk a few people told me about their applications, for which longer garbage collection pauses definitely meant longer response times and definitely mattered a lot. I certainly wasn’t going to tell them they were wrong, either! I think the number of cases where there’s a strong correlation between garbage collection pause times and response times - or, at least, maximum response times - is pretty high. What I was trying to get across is that there are also at least some cases where the gc pause times are only a small contributor to the maximum response times. It would be unfortunate to minimise pause times without ever checking back against response times if your application happened to be one of the ones where pause times aren’t so important.

The whole subject of performance tuning is fascinating because there are so many different system configurations and so many different performance requirements that coming up with a single answer is impossible. Even coming up with a single coherent set of heuristics is pretty tricky.

Six Myths and Paradoxes of Garbage Collection - MSc Thesis

Tue, 17 Apr 2007 00:00:00 GMT

This dissertation was written in 2007, so in tech terms, it can be considered quaintly antique - back when it was written, there was still a debate about whether garbage collection was a good idea. StackOverflow hadn’t been invented yet, and C++ was the language to beat.

What’s the dissertation about? Well, this is the abstract:

Many myths and paradoxes surround garbage collection. The first myth is that garbage collection is only suitable for the incompetent, unskilled, or lazy. In fact garbage collection offers many architectural and software engineering advantages, even to the skilled developer. The second myth is that garbage collection is all about about collecting garbage. Garbage collectors also include an allocation component, which, along with their powers of object rearrangement, can make a significant difference to application performance. Thirdly, criticisms of garbage collection often focus on the pause times, and responses to these criticisms often focus exclusively on reducing pause times, in the mistaken belief that small pause times guarantee good application response times. Pause times are also often used as a metric of general application performance, and an increase in pause times is taken as an indicator of worsened performance, when in fact the opposite the opposite is often true. Paradoxically, even the total amount of time spent paused for garbage collection is not a good predictor of the impact of garbage collection on application performance. Finally, the sixth myth is that garbage collection has a disastrous performance impact. While garbage collection can hurt application performance, it can also help application performance to the point where it exceeds the performance with manual memory management.

Although some of the arguments it makes are no longer necessary, there’s a lot I still like about the paper. It’s got equations, it’s got queueing theory, it’s got dissections of usenet trolling about GC, it’s got diagrams, and - the best bit - photos of garbage bins. You can read the full 84 pages here; it’s licensed under Creative Commons Non-Commercial Share Alike.