The Random Ramblings

lost? dazed? confused? dont come here then as this aint goona help!!

Where DevOps fails.

There is a lot of buzz around DevOps these days with every company you can think of looking for some one to set up their DevOps environments.

I, as some of you might know, work as a DevOps lead. A little more glorious than it sounds, as with all start-ups, pretty much everyone is lead or senior as there are so few of you but it also means that you are then responsible for setting up that companies whole ethos and path for the future. Being in that line of work and previously being a contractor you get a great idea of what is happening in the market as you still get lots of emails from recruiters. Something of late that's had been coming across, from a very large percentage of emails, started to concern me. Lots of mentions of full stack developer, must be able to program in xxx and yyy with so many years experience, front end and ui programming knowledge......

Now there might be a very legitimate reason for needing full stack skills, front end, ui. For example the company might want some one to help build a tool for the DevOps market. It might be that the no longer have a development team, everyone is under the DevOps banner, there are those that focus on code creation, those that focus on systems but you are all one and the same and the running of the application is equal responsibility, none of that chucking it over the wall, something that still exists in some DevOps environments I've worked in, in the past. I really do hope that's the case. Sadly I don't think so.

I will be honest, certainly right now I am not a developer. Yes I write code and yes it works, sometimes it works correctly!!! That does not make me anywhere near being a developer. I know devs that can install apache, run mysql servers but sadly and many will deny it, but that does not make them capable of being a sysadmin. It's a very different mindset and a very different train of though. More importantly there is a lot of years experience and normally of tools that are not often mentioned because they are considered standard and then there is the fact that we have often been trough the pain, those days with so little sleep due to DR or system crash where things need to be rebuilt or recovered and nothing works like it should. I suspect that devs have the same issues, I've thrown code back because it didn't run in production and they have had to find out why when it runs perfectly on their system. Getting round this is where DevOps is such an amazing ethos as it should allow the combination of skills to get a better understanding from each side but also to build things together across the application/system from dev to customer. Still caution and to the start of this post.

Just recently, the same day two situations, nothing that directly affected me but one very close and the second with a good friend. The first involved encryption, testing and people saying it was all fine and ready to roll to production. Luckily for all involved the ops man decided to just run a quick check against staging so he was sure that what was about to be rolled into production was going to be configured correctly and what kind of response to expect from the logs. He fired off an upload and watched the packets, instantly he saw one packet contain something that looked like an offset of what he sent, closely followed by what he had sent in plain text. Concerned that something had changed he checked and everything was configured exactly as specified by the team that had tested. Long and short is that neither person that was part of the implementation or testing team really understood wireshark or tcpdump. Both are excellent and long served developers at the top of their game but neither had ever really need to check traffic in that way previously.

The second was a piece of software, for rpm systems, team run DevOps according to their website. This one sits in the certain realms of missing an ops who would have shouted loudly at them all and removed their release privileges but also, I think, a young and excited team will to get the best and greatest version out to users. Generally in software release you have three numbers major, minor, patch. Patch is for general bug fixes, minor is updates, progress forward possibly a little break of api backwards compatibility and then major obvious big changes and potentially a bit of work to move forward. When pushing to a repo, you only allow auto upgrade if things can be done without user interaction, if that's needed a new major version comes out and it's then the choice to upgrade knowing the work that's needed. This product was released when not only did it have to be upgraded in a specific order but it also needed a complete database migration. I wonder how many ops people are now on the war path. I know the friend decided that, with the response he got from the development team and the outburst from others about this kind of thing happening before, it was quicker it choose something else, test and roll that out than have to fix the problems that were cause because of their specific upgrade requirements.

DevOps is an amazing thing and is helping move technology teams in the right direction, combined with many of the other methodologies it can help deliver so much more and in a quick and efficient manner but please please don't take the cost preference route. You will very rarely save money if you think that you get more if all your DevOps team are devs, ops might not appear to add value to the end product directly but, if they are worth anything, they are saving money indirectly to pay their way. Using kit more efficiently, reducing customer calls by keeping things running, saving your arse by making sure data never gets lost. Both sides are needed and both will make you great.

Find Me
Javascript learning