Egy újszülöttnek minden vicc új, így én a régi viccekre szakosodtam, azokat mondom el újra és újra.

Floorshrink diaries

Floorshrink diaries

The crow is a singing bird #3 – The Ugly

2026. július 26. - Floorshrink

part_3_the_ugly.JPG

Welcome back to Part 3 of the singing Crow series about AI. In this part, we will discuss the financial landscape and unintended consequences of AI. 

AI value creation – unrealistic promises 

The following chart shows money flow among the players in the AI value chain. 

part_3-_value_chain.JPG

There are entities who wear two hats, those with a skyrocketing cashflow keep pouring money into the bubble, while requiring the receivers of these funds to spend it on products or services produced by the investor itself. For example, NVIDIA and Microsoft are a key funding source behind Open AI, that in turn buy GPUs and cloud to train and run their services. 

There are two special players: Google is the only AI provider who is independent from NVIDIA, designing its own chips. They are strong enough to do it (they have been producing their HW since the beginning minus the CPU) and they are doing their best to preserve their cash cow search services, under attack by AI chatbots. The other oddball is Oracle, becoming a DC provider itself for Open AI (or anybody else). 

The state is a player, since both the US and China treat AI as a key component to remain (or become) the supreme power in the 21st century. Their actions fall into two buckets, being a significant buyer and funding agent, and sometimes intervening with export bans. 

  • Problem #1: the players who make any profit in AI are not the AI service providers but the chip makers (namely Nvidia) and the computing infrastructure providers (eg. Microsoft).  
  • Problem #2: As per Crunchbase the world's Most Valuable Unicorns in 2026 account for 3.8 trillion USD market valuation, out of which 2.3 trillion are AI related. All of them are generating gigantic losses with no real profitability in sight. These AI unicorns are worth roughly ten times Hungary's entire annual GDP. Something is wrong here. 
  • Problem #3: the investment money is often not real. It has a fancy name: circular deals. These deals – eg. when Nvidia funds OpenAI, OpenAI buys Nvidia chips - are very close to the round-tripping kind of vendor financing; which rhymes with the dotcom bust. 
  • Problem #4: the timeline is unrealistic. Neither the infrastructure nor the early majority buyers with sufficient budgets to buy AI services will be ready in time, thus the poster child AI players will stay in the red for another couple of years. This is not exactly what investors are dreaming about. 

Problem #1: Where is the profit? 

Let us have a look at the P&L of Open AI: Even if we take out the one-time costs related to the transition to be a for-profit company, Open AI’s business model is not sustainable, hence the sudden change in their pricing model.  

 part_3_revenue_waterfall.JPG

OpenAI spending hits $34 billion in 2025: Ed Zitron – Blockspace  

ExclusiveOpenAI Losses Increased Nearly 8X in 2025, With Spending Hitting $34 Billion 

 Let’s have a closer look at the revenue sources of Open AI: 

part_3_open_ai_rev_matrix.JPG

The dimensions of this matrix:  

  • who pays the bill, you as a private entity OR your employer 

 

  • AI as an efficiency booster OR closely related to the product 

A 20 USD flat fee Chat GPT subscription is a loss maker, depending on the usage intensity of the subscriber. Large corporate buyers with a vested interest in reducing their SW development cost and increasing their efficiency are the real target audience. (Anthropic is more reliant on them.) But these buyers belong to the early to late majority; they want references and proven value for their money. There is a natural break point in buying AI based coding assistance (agents), the cost. When AI providers switched to token (API call) based pricing, the related price tag increased manyfold overnight. 

Problem #2: unrealistic promises 

The income of AI startup CEOs is tied to stock value rather than actual profits; they have a personal incentive to keep investors excited, ie. they make public statements in Davos (and anywhere they find an audience) that hide the ugly fact that these companies are hemorrhaging cash. As Doctorow puts it: “The tech platforms are desperate to convince Wall Street that you love AI, which is very different from convincing you that you love AI.” The result is shown in the next chart: the current P/E ratios are like those during the dotcom boom.

part_3_ai_bubble_pe_ratio.JPG

am.jpmorgan.com/us/en/asset-management/institutional/insights/market-insights/guide-to-the-markets/guide-to-the-markets-slides-us/equities/gtm-forwardpe/  

It looks like a bubble, smells like a bubble, quite likely it is a bubble. 

Problem #3: the investment money is often not real 

I call it horse trading, but the fancy name is circular deals. There is a high tide in announced mergers and acquisitions: SpaceX - Anysphere (Cursor), Google - Wiz, Meta – Spree, IBM – Seek AI to name a few acquisitions and the shift of crypto miner’s attention to AI, like the TeraWulf – Anthropic or the Core Scientific – CoreWeave DC lease deals with a big BUT: most of these transactions are stock for stock, rather than cash. The chart below is from Bloomberg. It illustrates how the bubble is built: NVIDIA makes a multibillion-dollar investment in an AI service provider or in a datacenter provider AND gets back the same amount since the receiver of the funds places an order for NVIDIA chips as part of the deal. 

part_3_feeding_the_ai_machine.JPG

AI Circular Deals: How Microsoft, OpenAI and Nvidia Keep Paying Each Other 

Problem #4: the timeline is unrealistic 

Building a datacenter plus making it operational takes several years. This is especially true for building semiconductor fabs and especially large power plants that can serve these DCs. This means that all those announced new DCs with thousands of GPUs (and power requirements measured in gigawatts) can become operational in the early 2030s. But the hot money and the investor's greed is here today. The two timelines do not match, and you cannot compress the first one. 

Side effects 

Just like earlier technology breakthroughs AI will have unintended consequences:  

  • The most important is the decline of trust, one can no longer believe what he/she sees, or even if the person he/she is talking to is a human. We will need to come up with something stronger than the Turing test. Harari is right; the straightforward way could be if all AI entities were marked as AI entities, let alone AI generated content. The issue is that this is exactly what the creators of those entities and content want to hide. 

  • Potentially the most damaging side effect is “cognitive outsourcing”, when users ignorantly accept AI-generated outputs as their own thoughts, running the risk of losing their critical thinking and hands-on problem-solving capabilities that are essential to build expertise in any domain. We may become dumber. 

  • “Enshittification”: This term was coined by Cory Doctorow. His interpretation is multi-staged; here I concentrate on the outcome, the ever-growing quantities of low-quality digital content. The classic example is Facebook. The platform that served the Arab Spring in 2010 degraded itself to a pile of AI generated crap and a bunch of marketplaces by 2026. What is more worrying is that future AI models might be trained on this crap that would have a serious impact on their performance. 

  • Privacy became a fiction: a woman in Iran cannot take off her hijab (head scarf) in her own car without being noticed and punished for it. When you make a phone call, the state will know where you were, whom you talked to, and what you said. The bulk of the population is self-profiling themselves on social media or by using their smartphones. Big Brother is watching you and there is nothing you can do about it. 

  • AI will trim the current high status of the SW Dev. community. I recall an early morning in San Francisco when we visited the office of a big data unicorn. The office was completely empty except for a huge bowl of fresh fruit and one of the founders. (I also recall beer and ice-cream in the Budapest office of the same unicorn.) I am afraid these days are gone. Despite this I am optimistic about the future of developers due to the Jevons paradox. This negative impact of coding agents will be tempered by the increased demand for SW and human oversight of the code produced by AI.  
    The real issue is our inability to fulfil the later demand. As agent-based coding matures, we will not be able to cope with the amount of code or even understand it. We might become the man with the red flag in front of the AI based SW generating machine. Another unwanted consequence will be the lower demand for juniors. The caveat, nobody was born as a senior… 

The final word 

AI is already good for several use cases like image recognition, content creation, content analysis, speech to text and text to speech conversions and translation. With its current speed of development, it will be good enough for SW development within a year or two. Chances are it will surpass human limits in many areas of life just like it did with chess and go. At this point the decisive factor will be the price of these services.  

People with a vested interest in maintaining hype are touting extraordinary outcomes by tomorrow. This is unlikely due to the Solow lag and the inertia of key buyers. Since the financial promises to investors will not be fulfilled, the current valuation of AI-related stocks cannot be maintained. The burst of the current bubble is not a problem by itself, although due to its gigantic size this burst may generate a downturn in the world economy. The real thing is the long-term implications of this new technology that are unforeseen and quite likely unprecedented. 

Sources 

 

 

 

 

 

The crow is a singing bird #2 – The Bad

part_2_the_bad.JPG

Welcome back to Part 2 of the singing Crow series about AI. In this part we will collect the controversial aspects of AI.

AI – a force of creative destruction 

If textbooks are correct, the price of a stock represents the discounted present value of all expected future profits and cash flows the company will generate, divided by the total number of shares. If Anthropic and OpenAI are worth 1.8 trillion USD, then their future profit potential must be in the 10+ trillion USD range. This implies that their product will either create entirely new markets (worth 10 trillion USD, for reference the US GDP is around 30 trillion USD) OR will disrupt existing ones. You can find examples of creative destruction throughout the history of technology development: media streaming wiped out video rental stores, e-commerce ate the lunch of physical stores, digital photography killed the film photography, or going back in time: the same happened to horse drawn carts when the railways and later automobiles arrived, not mentioning steam powered looms that replaced hand weaving jobs. 

My theory is that the AI sector’s huge valuation derives mostly from the salaries of the human workers it aims to replace, not new markets. This means that it will have to substitute existing value creating tasks cheaper than they are carried out today by us humans. So, the question is not which industries, but which task types will fall victim to AI, and which ones will stay immune and why? 

Phrasing the question as a 2x2 matrix, we will end up with something like this: 

part_2_2x2_matrix.JPG

The dimensions of this matrix:  

  • the role of AI (augmenting or replacing human work) 
  • the consequence of an error made by AI 

There are two areas with little ambiguity: AI will be welcomed in the upper left corner of the above matrix and quite likely will be a “no go” in the lower right corner. AI will be used in designing a new drug, but a human will remain in charge of controlling its testing and granting the green light to release it. In general, fully blown automation will be blocked by the regulators in areas where clean accountability is required and where the social cost of an error is unacceptable. An existing example is modern passenger planes. An Airbus (eg. the A350) can land without any human intervention, but despite its capability the pilots are still in the cockpit. People just love to kick physical asses rather than entering an endless lawsuit against an unpunishable algorithm or its maker (another algorithm). 

The jobs (tasks) under attack will be those with the following characteristics: low risk, low accountability, high ROI (cost of AI used vs. the cost of equivalent human labor). This translates to junior level jobs (no accountability), little damage if the output is questionable (low risk), and labor intensive and repeatable (high ROI) On the other end of the spectrum are jobs that require physical manipulation of complex real objects. In a bon mot from Geoffrey Hinton – be a plumber.  

Beyond affecting jobs, AI will have implications for entire industries as well. 

  • If Guess can use an AI generated model in Vogue, then this model (or a thousand different ones) can play a role in a movie. AI based content generation is likely to cause similar upheaval to the movie industry like the talkies did in the 1920s, but this time nonexistent people might compete with flesh and blood actors. A less visible, but realistic case is polishing the scripts for the same movies. 
  • Over ten years ago - being responsible for training at an SSC – I found usage patterns on Udemy that were strange: people picked up knowledge in small bites: they consumed a 60 min training lesson in 3-4 pieces, not at once. Many of them abandoned the whole thing after finding the answer to a specific question. AI will give this trend a boost: it will remember what you wanted to learn earlier and can carry on a dialog with you while learning. If we consider that AI models know more about any given subject than most elementary school teachers AND that the average age of these teachers is around 50, (let alone there are more math teachers over 60, than below 30, at least in Hungary), the writing is on the wall: education will dramatically change in the next ten years. 

 

Forces that will hold back progress 

Innovation infusion is slower than innovation itself. The ultra-fast development in AI capabilities cannot translate to immediate economic gains. This is not new; it even has a name: the productivity (or Solow) paradox. As Robert Solow observed it: "You can see the computer age everywhere but in the productivity statistics." Background: While using computers became widespread in the 80’s in the United States, the increase in productivity just did not occur until 10-15 years later. 

  • The speed of progress hurts adaption: clients with deep pockets want to see a proven ROI on their investment and have problems with switching models let alone providers on a monthly basis. 
  • Cost: AI providers realized that their financial position was unsustainable, so they threw the monkey wrench into the works by moving to token-based pricing. The question is if there is an alternative, ie. If the high-end proprietary models can be augmented with good enough and more affordable alternatives let alone open source – open weight models running locally (uh-oh, did I say on prem?) 
  • Security: if there is a realistic chance that AI agents make ITSec shortcuts or plain stupidity while coding, the consumer of this SW product will want a certain level of human involvement in the coding or at least in the testing of this app.  
  • Regulatory worries about the fate of your data: Section 702 of the FISA (Foreign Intelligence Surveillance Act) permits the US government to conduct targeted surveillance of foreign persons located outside the United States, with the compelled assistance of electronic communication service providers. This means that US authorities have the legal coverage to read your data in an EU data center and can twist the arm of a US service provider to get assistance in this effort. If you add the recent moves by the current US administration to ruin the partnership with its allies (that took a few decades to build), it makes sense to create an alternative AI (and cloud) service for the EU. 
  • Accountability and explainability: if you hit someone with your car, you go to jail. What if your AI (self) driven car hits the same person? Or an algorithm rejects your credit application, and the bank employee has no clue why. (try to "defrost” your account at Revolut who runs without any human based call center) 

 

AGI will not be achieved via LLMs any time soon 

Language prediction is not the same as understanding. Scientists achieved great discoveries by observing and understanding the external physical world around them. An LLM model will produce a correct description or even an explanation of gravity without understanding the physical model of the phenomenon it talks about.  LLMs are trained on the results of earlier discoveries produced by humans, not recognizing these on their own, in plain words their knowledge is second hand. 

There is one caveat to the above thinking: an AI learning from feedback in a repeated fashion (in a loop). A current AI model plus image and sound recognition combined with persistent memory and the opportunity to learn while interacting with the physical world (perhaps embedded in a humanlike body to camouflage its nonhuman being) might render my argument obsolete in a few years.  

The natural inertia on AI experimenting on the physical world will be the speed the latter can respond to new inputs. Imagine an AI contemplating new molecules for curing cancer. A new compound must be built, given to a patient (preferably to a mouse as a first trial), a placebo given to another patient, then waiting for the effect, then modifying the molecule and starting all over again. (ie. clinical trials). Interestingly, when a crisis occurs, regulators do switch gears. The mRNS vaccine against COVID-19 was approved within 11 months vs. The usual 10+ years. 

 Summary of Part #2 

AI will make a dent on jobs that carry little risk if something goes wrong, will substitute the lower end of the spectrum while keeping the seniors to put the blame on someone if something does go south and will focus on jobs that make financial sense to be replaced. 

This whole change will happen gradually because of regulatory pushbacks, high cost, the fear of buying it from an unpredictable country, the demand for accountability and explainability, and the internal inertia of the buyers.  

The crow is a singing bird #1 - the good

part_1_the_good.JPG

Prelude 

I have seen podcasts and essays about AI from high-flyer nightingales that were oversimplified and in certain aspects simply mistaken (eg. Kraftie #65). I do believe the crow is a singing bird, (albeit graduated from evening school), so she has the right to sing. I took the risk of being wrong and created my own forecast for the next few years. 

Part #1 - The Good 

  • AI will enable people to achieve more, just like the Centaur Chess created by Garry Kasparov, rather than man serving the machine as described in the Reverse Centaur theory by Cory Doctorow. It will be infused into all walks of life like gene editing, fabricating new drugs, robotic surgery, humanoid robots or agricultural drone swarms. It will be a coexistence between human and artificial intelligence to cope with problems that are beyond our current capabilities. The doomsday predictions about all of us becoming servants of an AI overlord look good in Matrix like movies, but they are not realistic. 
  • Real novelties will come by combining innovations, not AI itself. It will be “just” an enabler, like many other great general-purpose discoveries in the past. 
  • SW development will be democratized, not by the current low code thingies, but by agents co-authoring real code in regular languages. This will lead to automated and algorithm driven solutions in areas (and at release cadence) unfathomable today. The reason why I put this into the “Good” category? The Jevons paradox will kick in, as more areas of our lives will be infiltrated by software, the demand for SW engineering may even grow, not shrink. 

Part #2 - The Bad 

  • The idea of creative destruction is not new; it was described by Joseph Schumpeter in the 1940s. For example, Google did it to the printed newspapers by sucking out advertisement revenue, and as a side effect - in lock step with social media - killed quality journalism. AI will have an impact on many industries, destroying some of them while opening new ones that were not imaginable before. Disruptive innovation, no doubt, and most likely unavoidable. 
  • AI is not immune to the Solow paradox; its impact will not be as immediate as touted. Nevertheless, this change will happen but with a certain delay. The big unknown is the lag time. My guesstimate is 5+ years. 
  • AGI will not be achieved via LLMs (sorry Mr. Kurzweil). Well, it may not even be a bad thing. 

Part #3 - The Ugly 

  • AI unicorn valuations are based on an unrealistic promise, the big names’ combined valuations approaching 1.8 trillion USD… while generating gigantic losses. The combined market value of the “magnificent seven” make up 35% of the market capitalization of S&P 500 and S&P 500 being 40% of the total equity worldwide. Their PE ratios are like those during the dotcom boom. (These folks are not defecating gold either.) There is a bubble and it will burst soon, not a big deal, it happened before. (Unless you are an investor who will lose his shirt in the crash.) Mountains of secondhand GPUs and DCs will fall in the laps of the real giants or Uncle Sam. 
  • AI will trim – some would argue normalize – the income of the SW Dev. community. This negative impact of coding agents will be tempered by the increased demand for SW and the likely need (in regulated industries a mandate) for human quality control.  
    The real issue will be speed (or the lack of it) on the human side: as agent-based coding matures, human oversight will not hold a candle in speed or even deciphering complex AI generated code. I used to nudge our IT Ops with the UK Locomotives Act from 1865, that they – with the manually opened, ticket-based processes - are like the man with a red flag walking in front of a self-propelled vehicle (IaC based infra deployments), slowing down SW development. What if we humans will become this man with the red flag in front of an AI based SW generating machine cranking out code at unprecedented speed? 
  • I skipped any military applications of AI since I have no information about this field. I guess it will belong to the “Ugly” section. 
  • AI will have unintended consequences: The first is the decline of trust, when one can no longer believe what one sees, or even cannot tell if the person he/she is talking to is a human. The second (and potentially the worst) side effect is “cognitive outsourcing”, when users ignorantly accept AI-generated outputs as their own thoughts, running the risk of losing their critical thinking and hands-on problem-solving capabilities that are essential to build expertise in any domain. We may become dumber. 

I realize that some of above items could fit in the Good and Bad categories at the same time, nevertheless I wanted some structure. The question is when the above forces will reach a new equilibrium, when AI crosses the chasm and becomes mainstream. My hunch is this will take 5-10 years and may be cajoled by geopolitics. 

The rest of this post is an explanation for this forecast. I cut it into 3 parts to make it more digestible for people with little time. So here comes Part 1 of the Crow song. 

Part #1 – the Good 

AI will enable people to achieve more 

One day I was asked to write a summary of a large-scale cloud platform. (IaC used to build everything, NIST/ISO compliance, SDLC support, all the bells and whistles). The client desperately wanted it by the next morning, so I spent the better part of the night writing it. A colleague of mine threw the result into ChatGPT and sent back his suggestions for improvement. Many of his (Cheat GPT’s) feedback points were valid, achieved in a few minutes.  

I accepted the challenge and started to experiment with writing snippets of Powershell code in Opus, then – being pissed off with a poorly formatted CSV file produced by a cloud monitoring system, I gave the whole thing to it and asked to fix it. It went beyond my expectations, extracted the pivotable root causes from the comment fields, and produced an analysis of the outcome, let alone when I gave it another CSV report from a later period; with a few lines of prompt, it produced a time series comparison. The difference can be depicted like this:

part_1_old_battle_horse.JPG

AI is not necessarily a sinister tool in the hands of the management for getting the same tasks done with fewer people (AI washing and related careless layoffs already backfired) but a way to help people to handle more complex work or to endeavor into new areas of the IT domain that did not even exist ten years ago. (Perhaps the two sides of the same coin will coexist.) 

Real novelties will be combinations 

Real novelties will come by combining these innovations. An example is your smartphone: this is the convergence of amazing achievements like the microprocessor, the LCD display, the lithium-ion battery, a modern OS with touch UI, CCD based imaging, IP networking and VoIP, the world-wide web, GPS with digitalized maps and lately AI based image and voice recognition – each worth a Nobel prize or a Turing Award, in a small package in your hand. 

SW development will be democratized 

One of my pastimes is designing geometric bodies by coding them in Open SCAD, then printing them with a 3D printer. The “flower” on the left took me cca. 3 hours to build. One day I found this thing on the right in the printer: I colleague (copyright Balazs Bodnar) just finished building a scale model of his sailing boat, starting from a 2D drawing, ending up with 5 parts of the boat (since it would not fit in the printer as a single piece, see the STL file in the middle). The catch? The entire workflow (the 3D CAD design from the 2D drawing, chopping the hull into pieces, creating the pins and holes to attach them together, accommodating the rudder and the mast) was done without writing any code manually; it was done by Fable (Anthropic's latest model) in less time than my little thing on the left. Something has changed, and we surely do not see the final state yet. 

part_1_the_boat.JPG

A few days ago, a friend of mine showed me his new SW dev. project to assist (partially automate) with the creation of functional and NFR definition of IT Services at his workplace. He demoed the app, presented the source repos in GitHub (typescript at present) and the cloud backend. The big deal: my friend stopped coding decades ago and moved to ITSM, giving up competing with the younger generation. Not anymore. We picked up the gauntlet, youngsters! 

Summary of part #1 

AI is already an amazing efficiency booster for old folks in IT, leveling out the playground when it comes to a competition with the younger generation. It will democratize SW development, that will allow people to produce working prototypes without the daily hands-on experience in a given language. Controlling the quality of this code and injecting the non-functional requirements is another story. Quite likely it will be solved via automated testing. The real big thing will be when the AI enabled pieces will be combined to create something completely new.

Please unload your gun before entering - javított kiadás

the_prodigal_robber.png

 

Több visszajelzést is kaptam a múlt heti szösszenet kapcsán, amelyek rámutattak a post hibáira. Ezúton is köszönöm az észrevételeket, íme a javított változat.

A kulcs észrevételek:

  • A posztban összemostam két különböző kockázatot, ti. az AI önmagában hordozott kockázatait (pl. politikai befolyásolás, nem transzparens algoritmus alapján való automatizált döntéshozatal), és az a kockázatot, hogy az EU lemarad az AI versenyfutásban.
  •  az AI mellékhatásait épp olyan nehéz és költséges utólag kezelni, mint ahogy egy IT infrastruktúrába utólag belebarkácsolni az ITSec szempontjait. (touche)
  •  a belépési küszöb valóban leesett az AI felhasználói oldalán, de a modellek tanításához szükséges chip- és adatközpont-kapacitásnál ez még nem történt meg. (továbbra is vagyonokba kerül.) Ez a nukleáris célokra alkalmazható urándúsításhoz hasonló szűk keresztmetszet: kevés gyártó, követhető szállítási lánc. (lásd Stuxnet)
  • a „megállíthatatlan" és a „kontrollálhatatlan" nem ugyanaz. A fejlődés iránya tényleg megállíthatatlan (ebben mindenki egyetértett), de a sebesség és a forma még alakítható — pl. a Red Flag Act sem megállította az autók elterjedését, csak átterelte a fejlődést Németországba.
  • ha elfogadjuk, hogy az AI a nukleáris fegyverekkel összemérhető kockázatot jelent, akkor azonos szinten kell és lehet is azt kezelni.

 

A fentiek alapján a korrekcióim:

  • Fenntartom, hogy az EU-nak oda kell tennie magát az AI területén, mivel amit nem te építesz, azt nem tudod formálni sem. Ha az EU csak szabályoz, de nincs saját frontier training DC kapacitása, modellje, mérhető létszámú felkészült szakembergárdája, akkor csak szabályelfogadó, és nem szabályalkotó lehet.
  • Az AI-ra is igaz a „kutyaharapást szőrével” bölcsessége: az AI offenzívát nem lehet megállítani, de a válasz nem a tiltás, hanem a védekezés felfuttatása ugyanazzal az eszközzel, azaz igenis EU szintű támogatásra van szükség a témakörben. (nincs is szebb annál, mint amikor egy rosszindulatú AI algoritmust egy jóindulatú kap el, azaz a automatizált védekezés kötelező, bár a támadónak helyzeti előnye van.)
  • A technológia változási sebessége valóban követhetetlen a szabályozók számára, ezért ne a technológiát kell részletesen szabályozni — az úgyis köröket ver rád, — hanem a kimenetet kell mérni és ezért felelőssé tenni az AI gyártóját. (itt van egy kérdőjelem a felhasználó felelősségéről, lásd a korábbi kés analógiát, de az egy másik post)
  • Az USA-Kína AI rivalizálásra érvényes, hogy bár mindketten a világhatalmi pozícióra törnek, és ennek érdekében elmennek a falig, ugyanakkor közös érdekük az AI feletti kontroll megtartása. A nukleáris fegyverek elburjánzása ill. annak szabályozása azt mutatja, hogy bár általános megállapodás nem valószínű, de szűken vett, a kölcsönös pusztulásra korlátozódó viszont igenis lehetséges. A kérdés persze a deklarációk mögötti tettek verifikálása lesz, ui. az AI esetében nem elég egy szeizmográf a sunyiban elkövetett atomrobbantások detektálásához.
  • a társadalmi szerződés felrúgásának folyamata és következményei igazak az AI nélkül is. Azaz nem AI-szabályozással kell kezelni, hanem a munkaerő átképzésével, és előbb-utóbb az elosztási mechanizmusok újragondolásával (lásd Universal Base Income).

 

The memoirs of Kilgore Traut: Please unload your gun before entering

 

pls_unload_your_gun.jpg

Az AI - karöltve a robotikával (test és lélek 2.0) - megállíthatatlan, miközben az EU pont erre tesz kísérletet azzal, hogy szénné regulázza. Ennek a megközelítésnek a várható következményeként fokozatosan veszíteni fog a versenyképességéből a másik két pólussal (USA és Kína) szemben. Így kevesebb jut majd a jóléti állam szolgáltatásaira (egészségügy és nyugdíj), ami társadalmi feszültségekhez vezet, mivel az EU (öregedő, ie. egyre drágábban szervizelhető) népessége ezeket a születési jogon járó juttatásnak tekinti. Ezek a feszültségek a populista szélsőség malmára hajtják a vizet és végső soron az EU széteséséhez vezethetnek. Az alábbi blog post a miérteket gyűjti össze és amellett érvel, hogy az AI-t és a robotikát támogatni kéne annak érdekében, hogy az EU versenyképes maradhasson (nota bene ne essen szét) az AI-humanoid robotika okozta átrendeződés után.

Néhány napja végig hallgattam egy banki AI Security konferenciát. Az előadások szűrlete egy mondatban: „óvatosan az AI használatával, különösen forráskód feltöltésével, mert azok jó eséllyel a rosszfiúk kezébe kerülnek majd, inkább vágjuk vissza, abból baj nem lehet.” Rá nem sokkal egy AI fejlesztés kapcsán a compliance szakértő egy több oldalas dokumentumban fejtette ki, hogy mi-mindennek kell megfelelnie a kérdéses alkalmazásnak még mielőtt egy sor kódot írtunk volna. (GDPR, AI Act, ISO 42001, DORA, ISO 27017/27018 stb.)  A fentiekről egy Mississipi beli pawn shop-ajtaján látott felirat jutott eszembe: “Please unload your gun before entering”. Elképzeltem, ahogy a rosszfiúk kitárazzák a fegyvereiket, tán még a símaszkot is begyömöszölik a tatyójukba, mondván, “akkor ma itt nincs rablás.” Az a gondom, hogy az EU-s szabály hegyek lelassítják az AI és a társ területek fejlődését, miközben a két nagy ellenpólus padlógázzal előz meg minket, nem beszélve a rosszfiúkról - mondjuk Phenjan-ban - akik nagy ívben ignorálják az egészet és okoznak majd egyre nagyobb kárt az AI felhasználásával.

Az erők, amelyek nem engedik, hogy az AI-t bárki kontroll alatt tartsa

  • Elindult a legújabb kori fegyverkezési verseny. Minden nagy eredményt megelőz egy trigger. Az USÁ-nak ilyen volt Pearl Harbor vagy később a Sputnik, amiből az atombomba ill. a Holdraszállás jött ki válaszul. Kínának ugyanezt jelentette az Alpha Go győzelme Lee Sedol ellen, de főleg egy évvel később Ke Jie (a világ akkori legerősebb Go játékosa) legyőzése szintén a Deep Mind AI megoldása által. A kínaiak felvették a kesztyűt. Ennek a versenynek egy jó indikátora a bejegyzett szabadalmi kérelmek számának alakulása.

    Apropó fegyverek: Az autonóm drónok, de főleg drón rajok igénylik az AI-t mind a cél azonosításhoz, mind az terepakadályok elkerüléséhez. A túloldalon ezen támadások elleni védelem szintén nem képzelhető el a hagyományos - emberi - döntéshozatali mechanizmusok mentén, egyszerűen nincs rá elég idő. Az orosz-ukrán háború egyik már ma is látható következménye a hagyományos hadviselési szabályok újraírása.

patent_applications.jpg

https://www.wipo.int/web-publications/world-intellectual-property-indicators-2025-highlights/en/patents-highlights.html (EPO: European Patent Office) 

  • A második hajtóerő a pénz. Esetünkben sok pénz, elég megnézni az Anthropic (965 milliárd USD) ill. az OpenAI (852 milliárd USD) IPO előtt értékelését pl. Magyarország tavalyi, kb. 250 milliárd USD-s GDP-jével szemben. A technológiai fejlődésben lévő üzleti lehetőség egy új aranylázat indított el. A fránya kapitalisták az AI vevői oldalán is dörzsölhetik a tenyerüket: saját - szubjektív - becslésem szerint egy fehérgalléros munkatárs kb. 20% hatékonyságnövekedést tud elérni egy jó és jól használt AI használatával. Ez – azonos output mellett – minden ötödik ember elküldését eredményezheti a kicsikét intellektuális, de repetitív munkakörökből pár éven belül. (Lásd még Geoffrey Hinton elhíresült karrier tanácsát.)

    A hab a tortán az, hogy az AI legnagyobb vevője maga az állam, hiszen a „Big Brother is watching you” megvalósítása szintén igényli az arcfelismerést és a valós idejű hang elemzést. Kínában 700 millió kamera jut 1.4 milliárd emberre, kombinálva egy társadalmi kredit (social scoring) rendszerrel. Orwell megemelné a kalapját… London nem sokkal kullog a fenti arányok mögött. Avagy: „mindenki belepisil az úszómedencébe, csak van, aki a trambulinról teszi ezt.”

  • A harmadik erő az ego. A “megcsináltam” és az “én csináltam meg” érzés mindent visz. A Szilícium-völgyben nem az ”élni és élni hagyni” a mottó, inkább a Highlander-t nézik. (There can be only one). Ezek az emberek győzni akarnak, bármi áron. Ide tartozik a megfigyelés, miszerint a tudománytörténetben még egyetlen esetről sem tudunk, amikor valamit, amit meg lehetett csinálni, ne valósítottak volna meg.

  • A tudósok igénye a tudásmegosztásra - a számítástechnika története példa arra, hogy az emberi alkotásvágy fantasztikus eredményekre képes. (lásd Zuse, Atanasoff, Neumann, Shannon...) Az akadémiai működés hajtóereje a publikálás ill. az ezen publikációkra való hivatkozások igénye, aminek következményeként az áttörést jelentő eredmények mindenki számára hozzáférhetővé válnak, nagyon rövid időn belül. A nemzetállamok ezt nem tudják és talán nem is akarják korlátozni.

  • A társadalomnak szüksége van az AI által támogatott fejlődésre – nem a Kurzweil-i szingularitásra gondolok, még csak nem is a fúziós reaktorral működő, űrben keringő kvantumgépes adatközpontokra, hanem prózaibb dolgokra, pl. a gyógyszeripari kutatások eredményére, amik majd le tudják győzni az antibiotikum rezisztens baktériumokat vagy az arcfelismeréssel bíró humanoid robotokra, amelyek bemennek egy égő házba, hogy kihozzanak egy sérültet.

Miért gond, ha nem lehet kontroll alatt tartani az AI-t?

  • Az AI nem neutrális: Évekig Werner von Braun megközelítését vallottam, ti. a tudománynak nincs morális dimenziója, olyan, mint egy kés, más hatást vált ki attól függően, hogy egy sebész, vagy egy gyilkos kezébe adod. Az AI nem követi ezt a mintát, politikai töltéssel bír, ui. az erőviszonyok átrendeződését fogja eredményezni, ami az állam és az állampolgár közötti hallgatólagos szerződés felrúgását vonja maga után. Az állam nem tudja teljesíteni a vállalásait (mert nincs rá elég pénze), az állampolgárok egyes csoportjai pedig nem akarják betartani a játékszabályokat (mert megtehetik). 

  • Az AI demokratizálni fogja a károkozás képességét: pl. a Wannacry-t továbbfejleszti egy algoritmus és legközelebb nem hagyja benne a forráskódban annak a regisztrálatlan domain-nek a nevét, amin keresztül meg lehetett állítani az enkriptálási folyamatot vagy akár hetente új verziót fejleszt ki és - az okozható kárhoz mérten gombokért - piacra dobja.

  • A technológiai innováció elérése korlátok között tartása egyre nehezebb: a nagy áttörések kontrollját korábban azok költsége jelentette. (a Manhattan terv költsége az USA akkori GDP-jének kb. 0,4%-át tette ki és az Apollo program is hasonló GDP arányos költséggel bírt 20 évvel később.) Magas volt a belépési küszöb. Mára ez megváltozott: a kulcs területek (AI - humanoid robotika – 3D nyomtatás és a genetikai kód manipuláció) költsége exponenciálisan esik, ezáltal egyre szélesebb kör számára válik elérhetővé, beleértve a károkozást célzó felhasználókat is. Egy sufni laborban a korábbiaknál hatékonyabb (gyorsabban terjedő és gyilkoló) vírust legózhat össze egy maroknyi rossz arc tudós, vagy csak szimplán félre megy valami, mint pl. abban a bizonyos wuhani laborban 2019 őszén.

  • Az AI választásokat dönthet el: Orosz barátaink évek óta minden demokratikus választást igyekeznek befolyásolni a hamis információk terjesztésével és ezt mostanra AI által generált kamu profilok ezrein keresztül tehetik meg. (ez olcsóbb, mint egy troll hadsereg) Ehhez persze AI által generált deep fake tartalmat is gyártanak, (mert ez nagyobbat szól). A kellően felhergelt választópolgár aztán a megfelelő helyre biggyeszti az X-et… (lásd pl. Brexit, pedig az még „csak” data science volt Facebook frontend-el.)

  • A kulcs technológiai felfedezések blokkolása káros: ti. az állam meggyengüléséhez vezet. A Gutenberg féle Biblia 1455-ben jelent meg. Az Ottomán birodalom 1727-ig tiltotta az arab karakterekkel történő nyomtatást. Ezzel majd 300 évig meggátolta a tudományos eredmények széles körben való elterjedését. Bár sokkal rövidebb ideig - és kisebb kárt okozva – de ide sorolható pl. az angol Locomotive Act is (aka. Red Flag Act), amivel az angol parlamentnek (a lovaskocsis lobbynak) sikerült húsz évnyi versenyelőnyt adni a német autógyártásnak.

  • Az AI fejlődése gyorsabb, mint amit a szabályzók le tudnak követni. Edward Wilson megfigyelése: "Kőkorszaki emócióink, középkori intézményrendszerünk és isteni technológiánk van.”  soha nem volt még ennyire időszerű. A fejlődés exponenciálisan gyorsul, a jogalkotók nem ehhez a sebességhez szoktak, ők években, sőt évtizedekben gondolkodnak. (btw:  AI Act-je még nincs az USÁ-nak, de így legalább nem évülhet el.)

Adott tehát a 22-es csapdája: az AI-t korlátok közé szorítani nem lehetséges, ugyanakkor erre nagy szükség lenne. Abban biztos vagyok, hogy a féloldalas – csak a szabályozásra törekvő - megközelítéssel lábon lövi magát az EU. Gyalogos katonaként annyit tudok, hogy az EU-n belül nincs egyetlen chip gyártónk, op. rendszer gyártónk, felhő szolgáltatónk és AI szolgáltatónk sem, aki labdába rúghat világszinten. Az EU zálogháznak  mindössze egy táblára futotta az ajtón. Ez kevés lesz a rablók távoltartásához. Pár briliáns könyvben találtam megoldási javaslatokat ((pl. Mustafa Suleyman – The coming wave és Yuval Noah Harari - Nexus), ugyanakkor van két dilemmám.

  • Kb. úgy vagyok az AI-al, mint a Limitless c. film főszereplője, aki véletlenül hozzájut az NZT-48 nevű kísérleti pirulához. Az NZT az agy teljes kapacitását megnyitja — villámgyors tanulási képességet és páratlan összefüggéslátást — adva a főhősnek. Ezzel a cuccal a kivénhedt csatalovak újra hadra foghatóvá válhatnak (Hallelujah), csak a mellékhatásokkal kéne kezdeni valamit.

  • Az Inconvenient Truth c. film 20(!) éve már jól összegezte a klímaváltozás várható hatásait. Ugyanaz az USA, ahol ez a film készült, kilépett a párizsi szerződésből, ami a CO2 kibocsátás csökkentésére vonatkozó lépéseket tartalmazta. Az EU két kulcs riválisa fontosabbnak tartja a hatalmi kérdést, mint a klímaváltozás problémáját. Miért lenne ez másképp az AI szabályozása kapcsán? 

Ha van véleményed, kérlek ne tartsd magadban!

üdv Laci

Források:

 

 

 

Paulus in reverse gear

laszlok_ink_drawing_in_the_style_of_albrect_durer_depicting_a_m_6dee3ce6-9052-4f3a-a60c-e8ebbea90caa.png

Christmas was great because I had some time to read/listen (see the inputs below). I distilled the outcome into the following post. Something fundamental has changed with an impact on the cloud therefore I have to make amends to my previous thinking.

Background on me: I was so thrilled by the fall of the Berlin Wall that went to Berlin on my own money to celebrate its 20th anniversary of “Mauerfall” in 2009. I stood in the rain, watched the toppled dominos and listened to Lech Walesa and Miklós Németh. I saw this wall in the mid-80s as a symbol of being on the wrong side, so I was deeply moved on that November day. I believed in Europe as a concept and thought that the 20th century – when Europe, and Hungary in it screwed up so badly - was behind us.

I also believed in another concept: I have been a cloud advocate since 2012 and proud of being part of a cloud transformation at a local commercial bank. When I was asked to produce an exit plan for the new cloud platform, I wrote this: The more we use PaaS and SaaS instead of recreating traditional on prem IaaS, the less likely it will become that we will ever come out of it, unless we want to spend as much on getting out as we spent on getting in, that would kill the business case. There is no affordable exit from the PaaS (let alone SaaS) and there is no material need for it. I added that not using PaaS services (being prepared to leave fast and cheap) would be like taxiing on the tarmac with an aircraft but never taking off. I was convinced that the only real reason to exit was a geopolitical meltdown. I finished my case with pointing out that in case of a shitstorm Microsoft, Amazon and Google might be forced to suspend services under extraordinary geopolitical circumstances, quite likely in a coordinated manner. (on the same day). This whole thing seemed very unlikely. What I missed at that time that an exit does not have to be economically rational but might be strategically necessary.

A year later came the attack on Ukraine by Putin, then came Trump (term 2) letting down first the Ukrainians, then Europe (for the record: Europe was indeed a free rider for decades, piggybacking on the US for its defence, keeping its welfare states afloat by underspending on their military.). Then came the news about Venezuela, an “invitation to waltz” for any power who want to turn the table by force. Perhaps the geopolitical meltdown is no longer an unimaginable event.

Call it a confirmation bias but I interpreted the Draghi report last year as a proof for my case, that is the EU was inhibiting innovation with its overwhelming regulations, and this should be trimmed to improve competitiveness. I knew for sure that the EU missed the boat in several key areas, like chip design and production, public cloud let alone AI. We cannot even show up an EU Linux distro with a double-digit market share. (Ubuntu is from the UK...)

So here is the Catch 22 of Europe: US political volatility makes full reliance dangerous, while it lacks viable near-term alternatives and diverting from hyperscalers would lock it in stagnation.

The current leadership of the USA acts like a loose cannon, destroying partnerships that took decades to build. If we add that the US society is divided to the point that the foundation of liberal democracy may not be mended, let alone that social media on AI steroids makes it easy to hack an election, we might conclude that it is risky to place all bets on US technologies.

On the other hand, name the most important breakthroughs in the last 80 years of information technology and count the number of key innovations that came from Europe. If you go one step further: this is possible that the stagnation of the EU economies (combined with the aging and shrinking populations, plus the need to spend more on the military) will evolve into social instability as soon as the welfare state becomes unsustainable in societies in denial phase. (The fact that in Europe there are only small countries and countries who are yet to recognise that they are small is another story.) The bad news: IT innovation became very expensive lately; free competition turned into “techno feudalism” (copyright Varoufakis) with a handful of giants taking rents from anybody with a smart phone and internet access (roughly half of the world’s population) and here we are (that is the EU) with our pants down in the coming storm. We do not have the funds needed to catch up. We have no replacement options to US technology in the short term.

If we stay on US technology, we run a risk of being screwed, if we divert from it, we surely lose any chance to increase our productivity, therefore stay in the race with the US and China. The sovereign clouds are just band aids without an overhaul of the EU. Europe must invest in technology, act on the recommendations in the Draghi report (reduce regulations, build a unified capital market and focus on disrupting innovation instead of protecting what we already have) and it must swallow the bitter pill and re-prioritise promises on welfare services that were built during a period of growth and security that no longer exists. And we must do all of these pretty fast. In the meantime, as a bare minimum, we have to keep a copy of our own data on prem, encrypted with keys produced by us.

As always, I would be delighted to get your feedback on these thoughts.

Inputs:

  • Technofeudalism: What Killed Capitalism (by Yanis Varoufakis - 2024)
  • Kaput: The End of the German Miracle (by Wolfgang Münchau - 2024)
  • The Draghi report on EU competitiveness (2024)
  • Freedom - Memoirs 1954 – 2021 (by Angela Merkel - 2024)
  • Postwar: A History of Europe Since 1945 (by Tony Judt - 2006)

Related blog posts:

Redmond, we have a problem

 redmond_we_have_a_problem.jpg

It seems that the law of supply and demand doesn’t work the usual way in the cloud related job market: It creates behavioural distortions rather than the gradual move to a healthy equilibrium. While key players seemingly declared victory and shifted their sight to the next battlefield of AI, this anomaly, combined with the heightened scrutiny from the regulators might hurt adoption in the long run. This post aims to identify the root cause and to suggest possible ways out of this problem.

Symptoms - What’s going on here?

I have been tinkering with cloud implementations for several years. It baffled me that – despite of the cloud-dev(sec)ops engineer compensation being 30+ % above the average - the inflow of talent into this area is far lower than anticipated.

The salary structure for an individual contributor DevSecOps Engineer varies based on geolocation, level of experience, and company size. Below is a table outlining the approximate salary ranges for different levels in various regions:

pay_ranges.jpg

  • For the above reason my team lost 10+ top notch cloud/devops engineers in two years. Most of these folks went abroad, one of them as far as Vancouver. Some others stayed in their homes but switched to foreign employers.
  • Some contractors sold 80% of their time twice, to two different customers, one of them was so unashamed that he put his other job in his Linkedin profile. (There are telltale signs of this behaviour: insisting on full home office and missing regular meetings, later deadlines.) Some elevated this practice to the company level…
  • Some others played a fair game and told upfront that they work for multiple clients, carried 3 (!) notebooks (one for each client plus one for their own company, God bless virtual desktops…) and declared that they would not even pick up the phone on days assigned to their other clients.
  • The cloud IT market bears resemblance to the construction industry, two, sometimes three layers of subcontractors adding little value besides their margin to the price tag.

The root cause

The wheel reinvented: this is the imbalance between supply and demand. The thing that bugged me was that despite of knowing the impressive earning potential, less than one percent of the internal IT Operations workforce (in a HUN commercial bank) made a substantial effort to learn the new discipline. (Those who did soon left ITOps.) On the other side of the house few developers made a career shift to become DevOps/IaC experts.

Gartner found a good demonstration of the problem in 2021. They dubbed it IT Talent quadrant. This matrix uses the stack ranked demand (the number of job postings asking for a given skill) vs. the number of these job openings per candidate as dimensions and provides evidence that Kubernetes, Infrastructure as a Code and Automation are critical ingredients for any cloud implementation. For some reason Gartner did not update this chart since 2021.

it_talent_quadrant.jpg

I created my own explanation, the Commitment matrix, that uses the seller’s commitment to his/her employer vs. the buyer’s commitment to its employee as dimensions. (In some cases, the seller and the employee being the same.)

commitment_matrix.jpg

In most cases there is a gradual shift of any new skill from being a Spice to a Cornerstone and later to drift into the Majority. For some reason in case of the most wanted cloud expertise this shift is just not happening.

The reasons

  • the cloud is an expanding universe, more and more large companies make their inroads, thus generating new demand for experienced people.
  • Buyers would love to have Spice people on their staff, but are not willing to pay the requested premium, claiming that it would generate internal salary tensions. (or simply drawing the comp. ceiling too low.) On the other hand, the very same corporations are willing to pay twice as much for the same people as contractors.
  • Top engineers do not want to work for a large firm as rank and file, they pledge allegiance to their boutique consulting firms instead. Smaller size means a more direct connection of the person’ contribution to the performance of the firm, thus results in perks up to partial ownership, let alone being among great technical peers is a nirvana for an engineer.
  • Achieving Spice level requires extensive learning and practice. Let alone the industry dictates a breakneck speed: your knowledge will become obsolete within 4-5 years unless you keep updating it. Cloud DevOps and Security are good examples for the Pi shaped skillset. One needs to understand the traditional development principles like branching or a pull request (and must write decent code) while being familiar with the nitty-gritty of name resolution in a hybrid environment with private endpoints or how a policy set will interact with the underlying Terraform codebase.
  • The last item could come from etymology: Dev + Ops is like mixing oil with water. Development is akin to creating something new, thus experimenting with the unknown: little predictability with high level of autonomy. Operations on the other side hinge on high level of predictability and minimal autonomy. I already used the modified Wardley map to depict this divide, but it is worth repeating it.

modified_wardley_map.jpg

To make things worse top-notch developers disregard script languages and look down on the non-functional side of the house like a private DNS resolver or a cross-regional site recovery. My hunch: they do not care, let alone know much about these things and want it as a service. From time to time, I present on universities as guest lecturer. On one occasion I asked the participants (50+ BSc students in their graduation year) about the power consumption of an Intel server. No idea. How about a notebook: no clue. A hair dryer? One girl new it. Infrastructure is not sexy, not even when it becomes code.

Ways to handle this problem

There are multiple stakeholders in this game with multiple paths to follow.

Vendors - Reduce complexity

I picked Kubernetes as the veterinarian’s horse to illustrate the problem. People who dealt with Kubernetes and its automated deployment and configuration can attest that it is complex to implement and to run, even without its ingress headaches with private endpoints or a service mesh on top of it. For this reason, Microsoft has offerings like Azure Container Apps (ACA) or lightweight alternatives like Azure Container Instance (ACI) while allowing plain vanilla implementations on VM scale sets for masochists.

The downside is that this simplification comes with losing some of the configuration, security and monitoring capabilities. As a dreamer I wish we had a universal serverless compute resource on our hands like Azure Functions or Amazon Lambda. „Liberté, Égalité, Fraternité” for cloud computing: „Autoscaling, Resiliency and Security”.

Another approach is to hide the internal complexity altogether by moving to PaaS and in many cases to SaaS. This is exactly what Microsoft is doing eg. with items bundled into Fabric. The issue: the deeper you walk into the cloud forest (wandering to SaaS territory) the less likely you will ever come out. This reduction of complexity is not evil by definition, one could argue that it helps IT to create business value faster. But there is a catch: Once a senior executive of a large bank asked me what the biggest danger in cloud computing was. My answer was: if politicians on either side of the Atlantic go crazy. Two years ago, I meant it as a joke…

Service providers - Hide complexity

Complexity and skills shortage provide a business opportunity. In practice it means creating a layer between the offerings provided by the hyper scalers and their enterprise customers. This toolbox is a combo of blueprints for landing zones, IaC code base for cloud services, integration solutions for connecting the cloud instance with its on prem counterpart covering networking, identity management, service management and monitoring and automatically deployed policy sets to streamline compliance audits.

hide_complexity.jpg

While it has its short-term financial advantages to start each implementation from scratch (if you are selling this service), only the thin upper layer of customers can afford it.

Warning: your Spice people are your golden goose, and not just for the profit you make on their billed hours. Ignoring the need for or screwing up with the foundations will lead to flawed implementations that will haunt you either as a security breach or a hard to run environment that your client will hate.

 

Engineers - Thrive on complexity

The revolution in infrastructure platform arena (Software Defined Storage-Network-Compute, Infrastructure as a Code) is the marriage of two – earlier distinct disciplines. This is reflected by the compensation data for cloud architects and DevSecOps engineers on Glassdoor: this is in the 130k to 230k USD gross annual range in the US. The rule of thumb is that whatever a top-notch IT skill costs in NY or London, you will get the same for one third of this price in Budapest, voila, flourishing Shared Service Centre business. So, we are talking about 50-60k USD annual gross for a good cloud devops engineer or a cloud security expert. The emphasis is on good. There is never-ending debate about the relevance of certificates. I recall a top-notch colleague at Microsoft from last century, when I nudged him about his certs (the lack of them) and offered that I would cover the cost of any MCP exam. He literally threw his MCSD certificate on my desk in two weeks. (he left the country 10+ years ago…) If you are good, the certs are doable and a good advertisement, but true, certs themselves are not enough. So Folks, learn and experiment! Cost is not an obstacle, a Coursera (ex. acloudguru) subscription is 30 USD a month, time is the problem.

For the record: it is not all roses, as shown by the chart below.  (I found similar data for Hungary.) The IT job market is not that pretty as it used to be, but it this fact reinforces my previous mantra: learn and experiment to stay ahead of your competition.

job_postings.jpg

Another caveat is the industry and the location. Your compensation depends on the impact of your work on the outcome and the profitability of the sector you are operating in. The effect is a bit sad: healthcare and education could make a good use of top-notch IT if they could afford it.

profitability_vs_demand.jpg

Legend has it that when the famous bank robber John Dillinger was asked by a reporter why he always robbed banks, he replied matter-of-factly, “Because that's where the money is!” In the next chapter we will have a look at the second core problem with the cloud: hyper scale providers being greedy and siphoning out profit from the value chain.

As always, I appreciate your feedback.

 

Sources

 

Horseshoe bend #6: Galileo Galilei

galilei.jpg

This post is an attempt to identify the root cause of the apparent divide between the two major branches of IT and to offer a remedy to this problem. (ambitious, isn't it?) As always this coin has two sides, so I would like to learn the view of the Dev folks and IT Operations folks as well.

Contradictions

During the 2+ years of running the cloud transformation at a commercial bank I faced contradicting views on the following aspects of how IT could function.

contradictions.jpg

  • One extreme argued that the cloud is just another data centre, therefore it should be treated the same way as our own: same (ticket based) processes, same technologies (ie. nothing else beyond what we already have on prem) and most importantly same speed letting new things in.
  • The other extreme exclaimed that the cloud shall change most aspects of IT as we know it, we should replace the stop signs (approvals) with guardrails (policies), automate every aspect of our daily life and most importantly treat IT infrastructure as a product that we want to sell to our (internal) clients.

I recall when 28 years ago – being in charge of introducing Exchange 4.0 in a local commercial bank -  I attempted to explain to a deputy general manager that printing and faxing each and every e-mail he sent (in order to make sure the other party received it) was suboptimal and a read receipt was enough. (not kidding). I cared more for the consulting revenue than the rain forests, so I dropped the case. In the same bank I had arguments with the network folks that tracking IP addresses for every Windows desktop in a paper-based grid notebook is suboptimal compared to DHCP. I did not drop this one, gaining friends until they ran into issues due to duplicated IP addresses and the joy of troubleshooting them.  (then they relented…) It has been bugging me ever since why it took them so long to realize these things. Why is it so darn hard to embrace change?

The psychology of IT Operations

 

psychology_of_it_ops.jpg

One day a manager at IT Operations asked me how many times I recall when IT Ops was praised by the senior leadership for things going normal (rarely) vs. how many cases I remember when they were reprimanded after a major service outage. To be honest: for all of them. I had to agree with his point: IT Operations is strongly incited NOT to change what works since the bulk of issues are connected to changing some aspect of the service. Hence the need for a CAB (Change Advisory Board) in ITIL. The root cause for pushback against change is the deep belief that speed and stability are the opposites in the same dimension.

At this point I have to borrow a page from the book of Matthias Patzak, who in turn borrowed a page from Simon Wardley and tweaked his map by changing the vertical axis (visibility) to autonomy. Here is a modified Wardley map explaining why change agents are at odds with IT Operations. (a proposed remediation is on the chart)

wardley_map.jpg

The question is unavoidable: How can the infrastructure stay unchanged when everything that uses it changes at an unprecedented speed? My hunch: it cannot. The rest of this post is an attempt to prove this point.

The stakeholders’ view

The voice of the customer – In our case the app dev teams:

  • Putting the cognitive load on the customer of the service is a guaranteed customer satisfaction killer – when a developer needs to figure out the internal processes of the service provider. (eg. filing separate ServiceNow tickets for the VM, the OS, the RDBMS, the DNS entry, the domain join and the admin access) It is like Vogon poetry. (the 3rd worst in the Universe) Dissatisfaction is the hotbed of shadow IT. For the record, not just in IT: Ferruccio Lamborghini probably would have stayed with his Ferrari (and his tractor business) if Enzo Ferrari would have been a bit nicer to him or would have made better clutches.
  • Lack of speed and autonomy leads to disengagement. I recall a developer who wanted to test a new feature of MS SQL Server. It took him 3+ months to get a test server. By this time, he gave up on the whole idea he wanted to test in the first place. (He knew that the test bed he was asking for would have taken about an hour to implement if he was given a chance. But he wasn’t.) So, after 3 months he dropped the whole thing.

The voice of the business

  • The top management of companies are concerned about unforeseen changes that may have a devastating impact on the livelihood of their enterprise. Their worries are backed by data. The Corporate Longevity Forecast, eg. the time a company spends on the Standard & Poor 500 list is shrinking. In plain English even large established companies can disappear from the list or even become “also run” within a few years. (Nokia, Credit Swiss, GE, Qualcomm bidding for Intel, WTF?) The age of creative destruction is upon us: What worked in the past for decades may not be good enough in the next ten years.
  • Enterprises are trying to be prepared for and respond quickly to attacks from any new force in the market. The cloud is one of their bets. All parties but one agree on the following:
    a cloud transformation will deliver its value proposition only if the organization and the underlying processes are changed along with the technology.

When money talks - R&D budgets

  • If we assume that most Technology companies spend the same portion of their revenue on R&D and this R&D has the same impact on the bottom line (sometimes not true) than we may predict that more R&D (when it leads to a breakthrough), results in a quantum leap in profitability.
  • If a firm catches one of these quantum leaps in a life time, it is lucky. If it catches two, this has long lasting consequences for the entire industry. (Data points from 2023: IBM made 8.18 billion USD net income, in the same period HPE made 2 billion, Microsoft 86 billion.) The cloud race is over, the AI race has begun and the hyperscalers have more money to spend on it than their traditional competitors.

statista_stats.jpg

Source: STATISTA.com (data for HPE is from 2015 only, when they separated from HPQ)

I feel it in my fingers, I feel it in my toes (change is all around you…)

The following chart is a visualization for obtaining infrastructure for an app. Say the dev team working on App 1 wants an infrastructure with an application server with some compute power, an SQL DB, and OS and a VM underneath, plus this thing should be accessible via the web to clients. In a traditional org this would mean 5 separate ServiceNow tickets with manual handover between them. Eg. The virtualization folks would set their ticket status to done, a human being would intercept this change, and would file another ticket to the OS team to install the OS. These teams are measured on meeting their SLA-s, so they would close the ticket even if the client is not able to log on to this server. (After all identity management is a separate step, right?) Imagine a car dealer who tries to sell you an engine, a transmission, a few wheels and a body work as separate items, when you wanted a car…

the_tale_of_5_snow_tickets.jpg

In a cloud infrastructure it is a set of IaC scripts that ran at once.  And here comes the problem:

  • This automation could be built by a dedicated cloud group requiring an org change that is against the will of the existing org units. Injecting the SNOW tickets into the belly of the automation – with the same 5 day SLA-s - would require the same time as the traditional setup. If you are against the cloud all you need to do is to insist on sticking to the old process.
  • You can grant the right to execute this automation to the developer teams themselves, but it would mean relinquishing control and shifting to creating and maintaining the automation scripts and establishing guardrails (policies) instead of the stop signs.
  • Creating and maintaining IaC code, CI/CD pipelines and policies (some people might call it DevSecOps and Site Reliability Eng.) require new skills and could be seen as a threat for those not interested in the above changes.  

All in all, an innocent technology change proposed by the cloud would require organizational, procedural and skillset changes in an org who does not like change.

There is an interesting observation in the State of DevOps report for 2023. The more frequently you make changes, the more likely you will succeed. The root cause is simple: more frequent small changes (with a working rollback) touch fewer things that can go wrong. If we turn it around: the more worried you are about changing the platform, the more time will pass between changes, gathering more moving parts, that in turn will increase the likelihood that something will indeed go wrong.

number_of_changes.jpg

A side effect is that this will make your environment less secure (I will not apply that security hot fix because it might break the application – to be honest, sometimes it will) and will accumulate more technical debt.

There is an expression that is the tell-tale sign of a siloed organisation: “he is criss-crossing in my backyard”, read trespassing into a territory that the speaker considers his home turf.  “Any time you start something new like [an innovation – eg. the cloud initiative], that cuts across many areas, there’s a potential for people feeling like you’re in their backyard.” (Michael Britt) The problem is that most value creation process involves multiple departments, therefore one cannot innovate without “trespassing”.

I got into a conversation with the cloud transformation lead of a large commercial bank a few days ago. He made an observation that struck a chord: only a miniscule portion of the IT Operations workforce (in this bank) embraced the cloud, they honestly believed that everything was okay and this cloud thingy was unnecessary, so responded accordingly. I think Amara's law is at work here: "We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run." I am biased in this case, but I believe they underestimate the impact of the cloud and miss the opportunity to increase their market value.

Squaring the circle - the way forward

  • The known knowns:
    - the business hates when cost grows faster than revenue. READ: The days of extensive growth in IT staff are over. (if you care for the whys, check out "Red Plenty") There is one way forward: automation
    - what is likely that those willing to merry stability with speed will gain the upper hand vs. those who will stick to their guns and obstruct change.
  • The known unknowns: 
    - technology will create as many jobs as it will eliminate. (a recent study by Guardian suggests that it creates more than it destroys.) What is unclear which jobs will stay and which will transform to something new. My bet is that the mundane ones (repetitive ticket crunching) will fade, while those requiring more thinking (eg. designing those guardrails mentioned above) will grow their relevance.
    -large IT shops carry an enormous amount of legacy, applications that generate the vast majority of business value for the enterprise today. This is difficult to forecast when the above shift will happen and how long this shift will take.
  • The unknown unknowns: 
    - IT Operations can hold the business at gunpoint claiming that any org/process change will pose a threat to the current stability of the business, therefore any cloud adoption should happen on their terms and at a speed deemed suitable by them. The real unknown is how long IT Ops can resist the push from their own internal clients and the hyperscalers. (make no mistake: the stick will follow the carrot soon.)
    - for the record: while industry disruptors are already doing it, my prognosis that technology allows for speed while maintaining stability is not yet proven in large enterprises carrying a legacy. 

Famous last words: In 1633 Galileo Galilei had an unpleasant encounter with the Sacred Inquisition that forced him to recant his claims that the Earth moves around the Sun, rather than the other way around. After leaving the courtroom he murmured "Eppur si muove" ("and yet it moves") and spent the rest of his life in a house arrest.

As always, I will be glad to learn about your feedback.

Sources:

 

The memoirs of Kilgore Trout nr. 6: the elephant and the snake

Twelve years ago, I held a presentation at the Budapest University of Economics. I used a drawing from the Little Prince (when the boa constrictor swallowed the elephant) with slight modifications to illustrate the income over time curve. Warning: your government wants to keep you as a net contributor in the pension system, while you may want to have a few more good years. In 2012 it seemed like a funny thing, today it looks like a problem. Considering the likelihood that your pension will cause a significant drop in your standard of living, your goal is to push the blue milestone to the right while retaining (some of) your market value.

the_elephant_and_the_snake.jpg

The acceptance of the above curve depends on your age and financial status, but the first reaction usually is that this is wrong, “torque overcomes RPM”, experience rules etc, bottom line: the market is wrong. If you keep in mind “the customer is always right”, then you might become interested in the root causes of this devaluation and what we can do about them. If you are under 40, stop reading, if you are over 50, you might want to read on.

your_market_value.jpg

The components of your (job) market value

  1. Your experience – which doctor will you pick for a heart surgery for you kid? A newbie (who is eager to do it) or the 40+ year old guy with 15+ years of proven track record? The untold part of the story is that you do not want a 70 years old dude with a trembling hand to do this operation either. Ok, we are talking about IT, but keep in mind, Oppenheimer was 39 when he joined the Manhattan project… The problem double fold: your experience is amortized AND you are unwilling to let it go to make room for new skills and new experiences. You need to learn new things and learning gets harder as you get older. There is a potential escape route here: move to areas where the half life of your skills is longer, that is away from hard core IT towards something softer like process or project management or farming watermelons. The issue is that this area already got overpopulated with the refuges bringing the prices down. Another way is to move up in the hierarchy but it comes with the unavoidable and undesirable jostling for positions. (then hustling the pretenders….)
  2. Your network – to be precise a few key people in that network who act like your sponsor are vital to your career. These are the people who trust you, who put a bet on you and who will speak up for you in that vital moment when a decision is made about you (or not you). Side note: this is one of those things when size does not matter, quality does. And now the bad news: Like it or not, your network ages with you, that means that those who know what you are capable of might no longer be in the position to stand up for you.
  3. Your college degrees – I was a diploma collector once (3 university degrees). Then one day I asked myself when the last time was when I used a Fourier transformation or whether I could still use my coding skills in Z80 assembly. Diplomas in technology get amortized fast. The real value from those years is your capability to learn and the seeds of your network.
  4. Your language skills – whenever I meet an IT person who claims that not speaking English is okay, I lose my marbles. 90+ % of literature in information technology is in English… Bad news: the upper 25% of the new generation speak two languages before entering college. (The only area where I put a heavy demand on my kids was a high-level language cert in ENG and GER by the end of their high school. Okay, I also put some emphasis on math…)
  5. Your appetite for 60+ hours work weeks – being a workaholic is not a shame (been there, done that), albeit it will have consequences on your relationships with your loved ones. As the adage goes the only people who will remember that you worked that much will be your kids, not your boss. For sure this appetite will calm down a bit around 60.
  6. Your ability to learn and to forget – most folks accept the fact that the half-life of any technology related skill is around 10 years this means you will have to reinvent yourself at least 3 times during your active years. What many folks do not think about is that one has to “unlearn” the old ways of doing things in order to be able to absorb new things.
  7. The logical multipliers:
    • Your appetite for power – you cannot be a leader without starving for the right to make decisions. You will not be a great leader if all you care for is power and not your people.
    • Your health – although I accept the gene lottery idea, I think there are a few basic rules you need to play by: very little alcohol, no smoking, no drugs, enough sleep, lots of physical exercise and a wonderful woman (man) by your side.

Bottom line: to a large extent the market is right about reducing the market value of people over 50-55. On the other hand, they are wrong about rejecting old folks upfront without any consideration. I recall a disaster at Liptovský Mikuláš in Slovakia when a storm literally erased an entire forest in 2004 due to one thing: all trees in that forest were the same type, planted at the same time. Old trees are a must in any forest.  (pic below is my own)

liptovsky_mikulas.jpg

The cost side of the house

Homo Economicus beware I dropped minor things like inflation and mortgages, but I considered items like moving to a smaller home once you became an empty nester and inserted luxury items like a costly divorce into the mix.

the_cost_side_of_the_house.jpg

Houston, we have a problem: This curve does not look like a snake who swallowed an elephant.

What to do about this problem?

There is a gap between the income and the cost curve. If we accept the definition of happiness as minimizing the gap between one’s desires and one’s reality we have three choices:

  1. lower the bar of your desires and expectations
  2. stay on the job market longer and reduce the degradation of your market value
  3. increase the portion of your income from your savings

Option A is not that bad as it sounds. I have first hand experience about moving from a 6-cylinder BMW to a 3-cylinder Mini Cooper without any mental or manhood degradation. Fancy objects (cars, watches, gadgets etc.)  are not essential to your happiness, collecting excessive amount of them even suggests that you are compensating for something.

Option C is by far the best. The only caveat is that only a minority of the working population reaches “escape velocity” who do charity work only to save baby seals and rainforests. (besides being angel investors since they want even more money) OK, what about the rest?

salary_vs_return_on_investment.jpgSo here we are: the market is mostly right and becoming a follower of Siddhartha solves only a part of the problem. Here are the ingredients for preserving your livelihood over 55:

  • Drop anything superfluous from your life and use what you already have. This whole life thingy looks like a lease with an expiry date, ie. you will have to hand in all your belongings before leaving the stage.
  • Stop being concerned with everything. As Mark Manson put it: "Maturity is what happens when one learns to only give a f**k about what's truly f**kworthy." A subtler explanation is from Milan Kundera who described it as a choice about the number of mirrors you want to see yourself in. Accept yourself as is, minimize your social media activities and pick only a few people whose opinion you care about. The rest can go and fly a kite.
  • The final thing from my all-time favorite, the mother of COBOL, Grace Hopper: The most damaging phrase in the language is: “it’s always been done that way.”
    DO NOT continue doing things because this is how you did it in the past. Change in IT is inevitable let alone exponential. You need to adopt. It is like a winding road with curves where you need to change speed and direction to stay on it. 

long_and_winding_road.jpg

As always, I will be happy to hear your feedback and remarks. Happy riding, Folks! Laszlo

Horseshoe bend #5: Lessons learned so far

The following post is an attempt to summarize the learnings from our cloud journey in the first 18 months. You bet, this is biased, but it might help others who come behind us. Those ahead of us you may put your all-knowing smile on.

the_rocky_road_to_dublin_v2.JPG

How to go faster - the first steps in the chaos

Public cloud adoption is an intertwine of grassroot experimentation, the mandate from the senior management to establish an enterprise grade cloud presence and finally a crash landing of the first cloud workloads without a proper foundation. The sooner you have a program established around it, the less chaotic the first months will be.

You need a cloud strategy

that answers questions like:

  • why you want the whole thing in the first place, how and when do you declare that you reached this goal and what metrics are used to prove it. (eg. cost saving may not be a strategic goal, while speed is.)
  • what your core design choices are: cloud architectural design (eg. hub & spoke vs. VWAN), accepted building blocks (cloud services), CI/CD tool set (source and artifact repo, build and deploy tools), IT Sec key decisions (eg. rejecting the use of public IP, checking ingress code from the internet, policy layers, IaC framework and the toolset like Terraform vs. the cloud provider’s native tooling like Bicep) and most importantly a decision-making process how to reach these choices.
  • the question of ownership: Cloud is much more than a 3rd datacenter (in fact more than any other IT infrastructure), therefore its governance should be established in the context of Business IT, DevOps, IT security and IT Operations. This is not an ITOps internal affair.
  • The willingness to change everything: I could not find the source of this quote but I think this is true: “When digital transformation is done right, it's like a caterpillar turning into a butterfly, but when done wrong, all you have is a really fast caterpillar.” You have to change the processes and the org structure if you want to harvest the advantages of the cloud. Without these changes the result will be as slow as the original on prem counterpart is.
  • The right level of ITSec control – if too loose, you will be hacked, if too tight, nobody will use your stuff and shadow IT orgs will sprout out everywhere. You need to decide on a few core items:
    • single CSP, or multi cloud, distributed cloud yes/no, cloud native tools vs 3rd party for monitoring, managing, protecting it.
    • how far you are able (willing) to go with automation, mostly with Infrastructure as a Code (IaC). The dilemma is where to stop. The Pareto principle should give us guidance but it misses one key point: any manual intervention will defeat the purpose of the entire automation. This quote is from 1935, but it is as relevant as ever: “It is difficult to get a man to understand something, when his salary depends on his not understanding it.” /Upton Sinclair/
    • what your cloud operating model is: the conservative approach is when the dev teams file a SNOW ticket for everything in the cloud just like on prem, the avant-garde approach is when you give them freedom to implement their preferred PaaS component with their own IaC code and to go YBIYRI (you build it, you run it) for components that are not yet supported by central IT Ops.

Establishing the Cloud CoE

  • A program or an org unit: Management needs to find out if you are a project or an org unit. All peer connects (interviews with other enterprises who embarked on this journey earlier) show that introducing the public cloud at enterprise scale is a 5+ year program with likely evergreen residuals. Treating it as a project has implications, eg. 90+ % of the team will leave at the end of the program, taking all learnings with them.
  • Staffing:
    • #1: quick learners with a solid technology background are on high demand. Giving scraps of time of mediocre performers will defeat the purpose of the whole thing.
    • #2: the imbalance between supply and demand will crank up the prices to the point that can jeopardize the financial viability of the program.
    • #3: be prepared to lose your best cloud engineers to abroad. Our regretted attrition way over the internal FTE attrition. The replacement takes cca 3+ months. The ramp up will require another 3 months, ie. you are down with a top engineer for 6+ months.
    • #4: We underestimated, therefore understaffed the process, governance and compliance tasks. Cloud is not only an engineering task, but a heavy lifting on process and compliance, let alone a major change management undertaking as well. The non- engineering activities are 30+% of the job. (the process folks claim this is 50+%...)

Key decisions to make

  • what the public cloud actually is – a 3rd data center or something completely different? The CCoE was convinced that this is different while ITOps insisted that this was just another DC, therefore should behave like one: same technologies, same processes, and nothing else.
  • how far you want to go with self-service? One approach is to allow You Build It- You Run It where ITOps is not ready to operate the new technology. The advantage is that it will allow the dev teams to go faster but will require to build operations skills and capacity on their side. Another approach is to channel every cloud request into the existing processes and handle them as if they were an on prem request.
  • Some dev teams will want to tinker with PaaS components while others will want to concentrate on business logic and application-level tasks. In the latter case, centrally provided cloud services will be required for those who do not want to deal with the PaaS component operations. You need to define the boundaries between YBIYRI and these central cloud services (roles and responsibilities) AND need to establish this managed service layer. (this is mostly not a technical undertaking.) 
  • Drinking from the firehose - the balance between an R&D workshop and a factory - the number of PaaS services vs. the available offerings (let alone the Marketplace) Do not go beyond 10-15% of the total service offerings, otherwise you will be quashed by their quantity.

The forces that will slow you down

There are two forces at play here: ITSec and ITOps. (Compliance waiting for you around the corner.)

itsec_and_itops_1.JPG

  • On prem ITOps mindset will dictate that anything in the cloud should function just like as if it was on prem. They will demand the same technologies and processes, the same IaaS approach to anything. Their – legitimate – reasoning is that 95+% of the workloads are on prem today, therefore anything you create should look like the current stuff since it is easier to operate. The untold driver is fear that you need to address upfront: Nobody will lose their jobs but likely to have a different job (with a different skillset) within 4-5 years. All of us need to learn and unlearn.
  • ITSec requirements dictate technical solutions that take much longer in a bank than in a small (non-financial) account. It is like running the Marathon in a heavy diver suit while all others run in shorts… An example: in a public cloud cross regional DR capabilities come out of the box, unless you implement private endpoints when you lose most of this functionality.

running_the_marathon_in_a_heavy_diver_suit.JPG

  • The nose of the ship cannot travel faster than the back of the ship, ie. it does not really help to produce designs and technical solutions that other parts of the IT org cannot implement let alone comprehend. This is a lesson we learned the hard way: You need to move the entire ship. Trainings, constant communication, demos and regular small updates help the transition.

Dependencies

architect_in_the_spider_web.JPG

You will find (at least) the following dependencies:

  • Identity and Access management – the identity management process and technology. eg. your IAM system does not work with cloud native identities and/or it is being replaced therefore does not accept any changes.
  • Ticketing system – your team gravitates toward JIRA (as most SW dev. projects do) while ITOps will demand ServiceNow. Shoveling data manually from SNOW to JIRA is a pain in the neck but you want to track the hours in a single system.
  • Click-Ops - your IaC code will bump into manual steps in the process, eg. a FW port opening might take a week while your code runs for 45 minutes.

Technical issues

  • If you implement IaC you need to pay attention for the smooth coexistence between the IaC code and the policies on top of them. This is a daunting task to debug a code where both layers are in constant move.
  • on prem proxy servers and multiple firewalls plus an on prem DNS vs. your cloud internal routing design will give you a bunch of networking and name resolution issues where you do not have access to the monitoring logs of any of the on prem components. it will require a smooth collaboration with the network people to resolve simple issues like a wrong conditional access setting.

The exit strategy

There are 3 caveats with a cloud exit:

  • when you mix up a disaster recovery and an exit scenario. the difference is the RTO allowed. the first is measured in hours, the later in years. It takes the same effort to walk away from a cloud than to walk into it.
  • when you allow only technologies that have an on prem equivalent. This way you do preserve your exit but throw away any innovation produced by the cloud provider. The deeper you go into the PaaS/SaaS forest, the less likely it is that you will ever come out.
  • when the seller’s state, eg. the USA says NO. In this case a cloud-to-cloud exit becomes unattainable (MSFT, Amazon or Google will leave the local market on the same day)

A reasonable exit strategy should be formulated, that will be acceptable by the local regulator. Regulatory, compliance and engineering task forces should collaborate, with an experienced leader (the best is someone who worked as an auditor before). Think twice before you execute this exit. This will ruin the ROI of the whole thing.

The square peg in a round hole – the lack of public IP 

If we had to had to name one item that caused us the most headache, it is easily the fact that the public cloud is designed with the internet in mind, that is that all services can be accessed directly from the internet. In case of an enterprise environment this is not the case, you have to go private.

The nonfunctional requirements

  • All of these requirements are known for decades, but work differently in the cloud, especially for PaaS and SaaS. Think about monitoring, logging, alerting and backup early and make reasonable compromises with their on prem counterparts.
  • Cloud monitoring, alerting and logging should be incorporated into the company level monitoring, alerting and logging. It is inevitable because the cloud-based systems will not operate standalone but integrated with on-prem (and later maybe other cloud) systems. In case of a problem an end-to-end view is needed, and it is possible only with an integration between the various monitoring systems.
  • Backup: you need to have a clear view on what you need to “bring home”, ie. back to on prem and what is okay to store in the cloud. At the end of the day, it boils down to the level of trust in your cloud provider and the demands by the regulator. Be aware that some of the backups provided by the provider are not compatible with anything else, ie. cannot migrate them to any on prem equivalent. (eg. KeyVault)
  • The big shift is when the Application Operations teams will claim a bigger slice of the traditional monitoring and alerting pie, using their own – mostly cloud native – tooling that will overlap in functionality with the tools used by IT Ops.

The non-technical side of the house

We shuffled all non-technical topics into a single team: Process – Governance – Compliance – Cost. In retrospect we underestimated the amount of work and the difficulties related to these topics. (engineering myopia) In fact there is a significant difference between “it works from an engineering aspect” and “it is a service one can provide with a predefined SLA”.

the_real_x_wing_fighter_and_how_it_looks.JPG

  • ITSM processes: IT Service management processes assume that everything is done by ITOps, the client just files a service request. ITOps is right claiming that an incident is a pain regardless where it happens, therefore you need to have a proper incident (and change) management process. If you are an ITIL shop, you will find out that a big chunk of the areas covered by ITIL3 are simply not applicable for the cloud. (hence the introduction of ITIL4 several years ago.)
  • The cost thingy: This is very easy to leave the lights on (on prem “flat fee - we already paid for it” reflexes kick in) but will cost you dearly. IT is one thing to spin up resources automatically, and seems like just a small change in the code (create vs. destroy) to tear them down. But somehow it just does not happen without forcing it. This is not by accident that FinOps became a discipline on its own right in the last couple of years.
  • The service catalog: In case of a cloud request the client may ask for a subscription, then for the predefined set of PaaS components in it, or just for the subscription and then would do the rest him/herself. Ie. you need to clarify what the service catalog should contain.

What comes next

at_the_beginning_of_the_journey.JPG

I wanted to thank the entire team who walked along in the last 18+ months. We are not finished by any measure and with the quickening speed of change we may not even know what “done” really looks like. What is beyond doubt that the big players turned their attention to artificial intelligence. It is a safe bet to forecast that AI will infiltrate all aspects of the cloud within a few years and will become the new battleground.

To finish with some fun: I used Midjourney to illustrate this post. The last prompt I used was this: “the magician pulling the rabbit out of the hat but the audience is not happy, cartoon by David Horsey, --ar 3:2”. Is it possible that AI already went rouge?

ai_went_rouge.JPG

As always, I appreciate any comment of feedback.

 

 

 

 

süti beállítások módosítása