Often the word DevOps connotes cloud infrastructure and a set of tools and hence existence of the DevOps teams in many organizations.  This categorisation and organization is the projection of understanding existing tools and practices to an emerging category and an attempt to co opt or assimilate new tools and ideas. This categorisation fails to take into account the basis of DevOps movement.

In this talk, I will be covering the motivations and incentives for DevOps movement and why it is successful when it is applied with a first principles approach.

If you look at any activity, there are subtle motivations and tradeoffs which direct our actions. The idea of transaction cost for a given transaction is such a subtle backdrop for many of our seeming irrational actions. From farmers going to a loan shark instead of banks to the rise of nfts and tokenization. Ronald Coase back in 1937 asked a fundamental question: If the markets are so efficient, why do companies exist? The essay is an excellent read on how organic organizations emerge nudged by intrinsic forces like transaction cost. What Coase found out was the trust between two individuals is the most costliest of the resources.

The lack of trust increases the transaction costs of business transactions and a structure where trust can be established more easily, brings down the transaction cost and hence subsequently the cost of goods and services.

So how does this apply to DevOps thinking? In my talk I will show how we can leverage this thinking to reduce batch sizes and thus increase flow and effectiveness of the teams.

Enterprise wide data collection always has data quality issues. Those who have ever dealt in collecting productivity data during haydays of SEI-CMM would know what I mean. Top management would always focus on being as predictable as possible in operations. That means getting a good handle on a simple thing like how much time a task would take for a given technology in a given domain. So everytime something new came up, there will be more data points to be collected for more accurate forecasts. It would reach a point where the people who are supposed to capture this data simply would not have enough time to truthfully gather what is being asked for, apart from their routine day jobs. All the data collection requirements would be seen as a way by top management to dissociate from failure if any, by simply stating that they did not get enough data. This only served a political purpose.

This sort of approach is a typical feature of F.W. Talyorian approach where everything is looked at as a machine to be orchestrated by a competent person in charge. It excludes lot possibilities when the strategy meets execution. Namely, Knowledge Gap, Understanding gap and execution gap. So for any undesirable outcome, there will be more effort to be spent on collecting more information, more reporting etc. etc. discounting uncertainty in external environment

.But any of what I have described above is not even the most damaging part. The damaging part is the deluge of requirements force people to lie and managers to accept it as a white lie. Everybody knows that the data they are getting is rubbish ( as they themselves might have done when they were collecting data).This “strategic lying” leads to what Ann Tenbrunsel termed as “Ethical fading”. In her words “This paper examines the root of unethical decisions by identifying the psychological forces that promote self-deception. Self-deception allows one to behave self-interestedly while, at the same time, falsely believing that one’s moral principles were upheld. The end result of this internal con game is that the ethical aspects of the decision “fade” into the background, the moral implications obscured.”

In this talk apart from the points mentioned above, I will cover how following “agile processes” and “dev ops” culture would alleviate one from this “ethical fading”. To enable truthful organisation, the top leaders have to carefully calibrate and limited amount of reporting burden that can be put on subordinates. So essentially it boils down to having a fixed bandwidth for reporting requirements and management can only add any requirements by removing one of already existing if the effort taken for the new requirements is likely to breach the “fixed reporting bandwidth limit”. An approach taken by prussian military in 1870s and later on adopted by various high functioning organisations throughout the world.

Deriving abstractions for a given problem description is always challenging. No matter what your experience level is, a new scenario will always be challenging to model. At the core of the problem is something known as "Inverse Problem" i.e. it is often easy to observe abstractions in hindsight than in foresight.

The modeling that we do,deals with IT systems and people operating those systems. What transpires from it is that the observed behaviour of a system is always a reflection of interference of multiple responsibilities of that system. To give an example, a lock is a lock because of the key. Should we lose the key, the lock is just another dead weight. So when we verbalize lock, certain properties come to our mind and weight is not one of them. But the properties that we perceive as properties of a lock are indeed interactions between a lock and a key.

It is very easy to extrapolate the lock example for unknown scenarios. In such cases we struggle because we have no clear idea about the underlying components and their interactions. So our best attempts are guess work. And if it works for scenarios at hand, we should stop generalizing. By extension of this argument, we can see that we should not strive for universal abstraction but try and model based on certain heuristics and create a workable domain and leave our abstractions incomplete. At the same time, we should have the humility to accept that our abstractions may be wrong but we should be ready to change them if such a need arises.

To conclude, in the session, I want to highlight difficulties faced in creating abstractions and how to cope with such difficulties. This is useful from developers to system architects and even CTOs to some extent.

One person can only know so much: Even the most gifted individual has a limit on the things that he/she can respond to successfully. Highly complex tasks exceed an individual’s capacity to perform or understand. Modern endeavours are undertaken in teams. Recent Nobel prize to LIGO team is a classical example of it. With a well functioning team, its not oft repeated idiom of two plus two but more modern version is two multiplied by two. The advantage of working together is to get a complex task right, to be successful at making the right decision. The higher the complexity, the more specialists cannot be successful, but teams can be.

However teams brings in another problem: that of coherence. So essentially we need to identify boundaries and communication mechanisms inside team and between teams. In the talk you can learn about pro's and cons of this approach, but one thing is certain that Teams is the way to go.

Generally scientific approach is kept limited to only reductionist approach i.e. trying to understand the system by understanding its parts. Thats how we have been taught to think. However this construct fails in non trivial systems. Often the property of the system can not be reduced to any property of its constituent parts. To take an example “pressure” is not a property individual atoms/molecules but that of a gas. In any complex system it is important to identify relationship between the property of the system and property of its parts. This relationship is known as emergence in complex systems.

In this talk I will cover variety of techniques I’ve used to identify emergent properties. I will also cover difficulties in modelling abstraction and some heuristics that has helped me unravel knotty designs and truly understand “deep insight” aspect of domain driven design. I feel this topic is important in the context of functional programming as functions are touted “pure” abstractions: missing the point of interaction.