Anonymous Analyst

Let’s start with the confession. It comes a bit harsh and straightforward, will try to elaborate it later on. Do not get offended (at least yet). Okay, I warned you, so here it comes - I do hate software analysts. More specifically, I have serious difficulties understanding what they exactly do and how-come the result is valuable to anyone. What I do like is when people understand the domain they are working on, they are clear on what they are going to build and who needs it. So the problem is not in a position itself, but in what people understand they should do and the value they produce.

In a way too-many organizations analysts serve just an additional link in a long chain between developers and the customer. The idea that someone will analyze the problem, chew it up and then developers would have to “just” code it, looks great on paper but simply does not work. It is a bit chocking and too obvious, but the thing is, developers do the actual implementation. They materialize thoughts into the program logic. So they have to understand the problem and represent it in the form of source code.

What does it mean in practise? Developers should do the same job that analysts just did all over again. But it gets even worse. Program logic is very formal. If it is Boolean - it is either true or false, but never both. We use that very formal language to describe problems and domains we do not fully understand or just not meant to be described. We try to imagine ourselves in a future using this new system, operating with all the stakeholders and all being perfect.

Let’s consider this small mental exercise. Try to put into simple if-then-else logic how do you select what you will eat for lunch. It should work for any given day and person. Probably you won’t even start doing that. Even if you will narrow the problem to selecting either you will have a meat or fish today, it does not really help much. You can draw some abstract analysis, that will look great, but you will fail implementing the actual logic.

Another problem with software analysts - they never fail. Their part of the job is done before the development start, all the blame is laid on the developers who have to rediscover what analysis did not foresee. Just think about it! Whatever you draw is genuinely correct. If you separate the fantasy drawing from the actual doing and you have no understanding what programming is - you can never fail. Developers, on the other hand, are perceived as dream-crashers, incompetent and lazy.

structure of organizations

The root of the problem lies deeper. Every organization is structured to better suit its processes. The structure is secondary, it represents a model how things work right now. The paradox is that later on structure prevents organizations from bringing in new and radical ideas. And the more complex structure is, much harder it is to make a change.

Organizations that have analysts, developers, testers, project managers separated from each other mirror classical Waterfall Model. Yes, currently it is very trendy to be agile, so everybody does Scrum. Others who does not want to be related with agile for one reason or another do tell that they use RUP (firstly, usually it is an excuse not to mention Waterfall; secondly, they demonstrate rude misunderstanding what RUP is).  And such a model has its own downsides.

I keep on saying, that there is nothing wrong with one development methodology or another. As long as companies understand the consequences and are ready to pay the price. And none of the methodologies come free. If you are doing the waterfall - you are slow to react and have a lot of paperwork to impress your customer with, instead of software. If you are agile and/or value oriented - then you live a fast life of constant changes and having a trouble growing your organization as a culture.

However, it is much worse to believe that you do well something what is very trendy right now, but actually you are doing something else. And this is the moment I personally fail to understand. The problems that organizations face applying the waterfall process happen all the time - low quality of a product, no testing, late delivery, non-existing or wrong analysis prior development. Those problems do not happen once or twice. They reoccur constantly for many-many years in a row. Why? Why don’t they learn something from that?

The best part is, that in a sequentially structured organization it never feels like you are not producing anything. You are always delivering something, there is a lot of firefighting and a lot of movement. There is always something happens and someone need answers right this moment, or better to have them yesterday. And if you compare their process with something different, for instance - you actually do not need a test-and-fix churn at the end of each project, you might be better off without detailed upfront design by learning by doing. It is perceived as slacking off and non-professionalism. Their process is perfect. There is nothing to improve. There is no another way.

There is another way! No process is perfect, you always improve and always learn more about your people, your business and the environment you are in. If you believe Refactoring is good for the code, why don’t you believe the same refactoring will be good for organization? Chances are that you do not believe in refactoring, but then it is a topic for a totally separate blog post!

promise and failure of a bright future

Currently you might wonder, why do I express all my anger on analysts, while it is not only them to blame? Well, because they are in the sweetest spot - they cannot fail. Project managers get pushed for being over the budget, developers for missing the deadlines, but analysts are in the middle. Therefore, they do not feel responsible for what happens before and after. Their part of the job is done. They never fail.

Besides writing overly fuzzy and too long blog-posts I do teaching at the Tallinn University of Technology. In the autumn I participate in a course “Software quality and standards”. It is a obligatory subject if you are doing MSc in Informatics. From the whole course less than 10% have ever written a line of code, vast majority wants to become [drum droll] an analyst.

Same goes with the people who are coming for the job interview. Okay, many of them, who are actually invited to an interview, have written some code. However, quite many believe that when you start your career, you start in a Quality Assurance field. Then you study and grow and become a programmer. Then you have a chance to become a project manager. And here is the shortcut, if you start as an analyst, you can skip the boring stuff of actually doing the job and become a PM right away.

The result is, that projects are lead by the people who were avoiding software development per se from the start. And in such organizations, those people are responsible for negotiations with the stakeholders about the scope and deadlines, in some cases even for estimations. Then you wonder, why so many projects fail to be delivered in time and budget. (Not mentioning that the specializations above are not in one and same career tree)

When we talk about great project managers or analysts with great track record, we should not forget that people understand success differently. If there is a thousand of project managers in the country, the chances are quite high, that ten of them got lucky on every project and delivered in schedule. Or, maybe, organization just considers that situation when projects go over the line or completely redone as part of the business and able to pay that price.

And once again, this is what it always comes to. Is an organization able to pay the price that always comes with the methodology it is using? Probably, the better question is - what happens if suddenly it cannot? In any case, it is very depressing to work in a company where someone is constantly glorified and cannot ever fail and someone just has to make the work done, usually at nights.

flip of a coin

At this point you are probably convinced that analysts in organizations - might be a problem. It seems to be the perfect spot to make the twist in a script. But, let’s have another cup of coffee first.

The thing is, as surprising as it sounds - we do need software analysts. It’s just, the ones we have are doing the wrong thing.

Imagine we will let only developers to design and create a system for the customer without his involvement. Most probably, it will look like the visual representation of a database structure. All fields grouped together in a strict logical order. Massive search options to allow searching on every possible combination. No mixed views or additional functionality. Everything is on its place and nothing else is there. You can see People, Addresses, Phones separately, but you cannot see them all together.

But this is not what customer wants. He might say he wants it like that (because it makes logical sense), but then he will wonder if everything could be redone. The real use of the systems has nothing in common how the domain model or the database table structure looks like. People think about the usability, where we can do an action with a single click without opening ten-thousand other views before that.

That knowledge lies in practice and hands-on experience that developers do not poses and no one is able to specify it and write it down. That knowledge constantly changes as people try it out on practice.

This is where we need someone technical to think like a customer. Analysts should study the domain and understand it thoroughly. It should be crystal clear to them how the business works. They should help business to discover new ways how to optimize the existing process. They are technical enough to say that this or that solution is not possible with current technology.

Of course, ideally every member of the whole cross-functional team should be able to do it. In one way or another they should finally know as much as analysts do before writing any line of code. However, having a dedicated analyst in a team, who simply has more dedicated time (and probably skills) to do exactly that is always better.

It can be compared to the researcher’s or scientist’s job. Where people work the magic, create theories, study and then try the idea out on practice together with the development team.

Another case is when a company does not have an IT department and orders the development from the outside. Having an analyst there, who understands the nature and complexity of the domain and how existing systems work - is just a blessing. He can keep an integrity among different systems and able to talk technical language with the developers. In many cases that person can represent the departments of an organization and become a Product Owner for the developers. Who can make the decisions and justify them among others.

I cannot stress it strong enough here, but in a nutshell. Analysis is ongoing as long as development is ongoing. And during that, you do not design software, you design knowledge and make it available to all members of a team.

what if you are an anonymous analyst
The constant firefighting and crises in projects happen because people do not try their ideas on practice. If you add there over-confidence that somehow things will be different just exactly this time during this project - and we have a recipe for disaster. The best part - it is an endless loop and no one dares to break it. But this is exactly what needs to be done.

Here are a few points that analysts should do. And if you are not doing them, it is actually quite easy to change.

• First things first, stop thinking that your job is only to analyze things. No one is interested in drawings or crunched knowledge if you keep it only to yourself. That does not make you smart or more valuable if you cannot send the message across to your colleagues. Your primary objective is to share knowledge and make it easy and accessible by all members. Writing a document - is not a knowledge sharing. It creates a document that probably no one will have a look. It gives an impression of shared knowledge, but only makes things worse. The right question is - what do you want to achieve with that document?

• Leave software and database design to developers or create the common view together. Software development is an evolution. We begin with an assumption that we know how to start. Then we develop it as simple as possible and grow it bigger. You cannot consider what you do not know yet, before you actually tried it. Normalized table structures might look great on paper, but sooner or later have nothing in common with the reality, or performance requirements.

• Forget UML diagrams. Firstly, just a few analysts know how to draw them. Secondly, if you have something to draw - engage whole team into discussion. Come to the common ground and, if needed, document it then. Use whiteboard, markers, pens, it is a way faster than drawing boxes in some weird program. Do not be a hero, it does not mean you cannot design it alone by yourself, it means you will do it better together.

• Become part of the team. You cannot finish your job before development start. You should sit together, share the knowledge, communicate. If you do not know what developers are doing, you know way less than you should. Help the team to get information, help Scrum Master to remove impediments, or even act as a Scrum Master.

• Generate and design test scenarios. Learn how to make your scenarios executable. The best way to explain how the system should work is by a good test case. Tests are there not to find bugs, but to ensure that our stories keep on working over time.

• Forget emails. If you have something to share, just say it out loud. Communicate face-to-face with the customer. The human language is full of ambiguity, why shall we make it even more complex and add another level of misunderstanding.

• Share the success and sorrow together with a team. If you screw-up - you did it all together. Everyone let it to happen. If you succeed - celebrate it also together. Everyone has participated equally to get the result.

No organization will prevent you from doing that if you just start. Those things seem trivial, but really... should it be complicated?

Do not forget to share your experiences!