Software development infrastructure. Part 0x00. Introduction

Introduction

Software development isn’t about writing code. At least not only. I would like to share my experience with the non-code-writing part

Development cycle

It feels people are convinced software development works more or less like this:

And it’s roughly true. It does however get interesting when one gets into details of what happens in the lower left of that picture, between the developer and the product.

• How do you write code?
• Where do you store code?
• How do you track changes?
• How do you handle business expectations?
• How do you handle customer feedback/complaints?
• How do you verify that your project does what it is supposed to?
• How do you arrange work with your workmates?
• How do you construct the end product?
• How do you deliver it?
• How do you monitor your products?
• How do you fix things?
• How do you recover your system?
• How do you cleanup the bug spill?
• How do you maintain backups?

And so on, and so forth.

Obviously, I would not expect a fresh software development enthusiast to know these things. After all, they have just written a premium hello world app and should not be bothered with all the ugly boring logistics crap. Some of the senior ones still don’t know this. And it still may be fine.

I’ve been doing this boring stuff ever since I started working in the industry. Although I am a dev, I have already done:

• Windows installers
• Build scripts
• Code generator pre- and post- generation cleanup/fixes
• Continuous Integration servers with integration builds, release builds, code analysis
• Project configuration in Ant/Maven/Gradle/Make/Bash/Python/…
• Version Control config/migration/integration
• Artefacts repositories
• Ticket systems
• IDE setup
• Code Style guides for the team
• Monitoring/alerting solutions
• Product deployment
• Product configuration
• Technical documents
• Integration tests
• Performance/stress tests
• Inspection tools
• Release management
• Backups management
• Recovery testing
• Build system plugins (Maven mainly)

And many many more.

I kind of like this part of work, but my relationship with it is complicated. I changed a job at least once because I would have my role changed to be solely responsible for that. I promised myself to never do that stuff ever ever ever ever ever again in my life.

And I’m doing it again.

The industry Sisiphus. Source: Pixabay

You see, coding is fun, making new things is fun, solving new problems is wonderful. Dealing with a system 10+ years old is slightly less fun. Dealing with something that “works so don’t touch it” can be a traumatic experience. Imagine all the hacks and shortcuts, uncommented changes from thirty different cases merged into a single blob of semi readable code in the distributed part of a system, without guarantee that you know all the tools talking to it. Sounds horrible, but it has two main advantages:

1. It works
2. It makes money

So you end up dealing with it, maintaining, improving etc.

Now imagine this code has no tests and most people who knew how it worked have left.

It can be a difficult experience, that’s why it is so important to have a proper software development infrastructure. In many cases it can be a lifesaver. In many cases it is only devs who can provide it to themselves, sometimes because the Operations team is overloaded, sometimes because the teams have freedom to do whatever they like, sometimes because the company adopts the DevOps, or DevSecOps philosophy, whatever that means.

So what

This leads us to what I plan to do: I would like to prepare a development/build/test/deployment environment for me to test things with and to learn. This will be a tutorialish series of posts so that you can benefit from it and point out where I am wrong.

Why

For fun. For exercise.

What I hope to get from this:

• Learn more Groovy
• Learn more Gradle
• Learn how to prototype and evaluate things
• Learn how to set up an artefact repository with backups and redundancy
• Get to know others how deal with such subjects and share experiences
• Get some questions

Plans

I am a lazy person with too much on my plate, so I plan to limit it to what I can benefit from at work.

• Open source tools only - I hope to be able to find something to contribute to, be it an issue or a pull request. While it's not the purpose of this series, it would be a nice addition.
• Java mainly – while many things can be adapted for use with other ecosystems, this is my thing for now. Apps will rather be simple. I will be using IntelliJ to make them
• Linux only – unless something changes in my life
• Gradle mainly – I’ve done a lot of Maven and I like it more than Gradle. But we decided to migrate our projects to Gradle so this is what I’m planning to focus on
• Jenkins initially – I’ve started working with Jenkins and it has its shortcomings, but then it has it’s power as well. I will be configuring jobs using Jenkins declarative pipelines as much as possible
• Nexus artefact repository – I know there are others, but Nexus handles things nicely and supports docker images that I want to test
• Git only – and possibly Gitlab; We use Gitlab at work. I would like to investigate how much effort would it require to integrate it with Jenkins the way Github or Stash/Bitbucket can be
• Docker – we use it for testing at work, so I know it already. It’s quite handy to set everything up with a docker-compose file
• Maybe some logging/monitoring/alerting
• I don’t know what else to do. Any ideas?

The first plans are to:

• Install Docker
• Set up Gitlab

What you may expect to see:

• Some of the above stuff
• Some bash
• Some Python
• Some of whatever pops up

Summary

What my expectations about you are:

• you should be rather comfortable with the geeky stuff. The terminal doesn’t bite
• you should be able to open a manual for a program in terminal and close it
• you should be asking questions. THE DUMBEST QUESTIONS ARE THOSE NEVER ASKED

See you in the next article, where I will set up Docker.

I have set up a git repository to host whatever files I may need in this tutorial. The readme also contains links to articles. The first code referes to part 0x02 and I hope to be able to release this soon. Till then, here'se a spoiler alert: you might lose some of the surprise.

---

Posted on Utopian.io - Rewarding Open Source Contributors

---