Why Can't We Have Good Documentation
Daniel Dib asked a sad question on LinkedIn:
Where did all the great documentation go?
In more detail:
There was a time when documentation answered almost all questions:
- What is the thing?
- What does the thing do?
- Why would you use the thing?
- How do you configure the thing?
I’ve seen the same thing happening in training, and here’s my cynical TL&DR answer: because the managers of the documentation/training departments don’t understand the true value of what they’re producing and thus cannot justify a decent budget to make it happen.
I got my first dose of reality when attending a beta-teach of Cisco’s Router Software Configuration course1. RSC was an excellent course that met Daniel’s requirements, but the new version was a watered-down affair, answering only “how do you configure that thing?”
Everyone in the room complained2, but the people running the course and collecting our feedback were not yielding a bit. Whenever someone mentioned that a good course should cover the technology fundamentals before going into the configuration process, their answer was “but there are generic technology courses out there, so whoever wants to know those details can attend those courses.”
That situation changed dramatically when a fed-up instructor said, “You know, more than half of my students attend the course because they’ve heard it’s a great technology overview course. Some of them might not ever touch a router.” You could literally see Ka-Ching go on in some heads, and the technology introduction was back in the course.
The sad part of that story is that the right decision was made for the wrong reasons – the potential increase in the sale of the student kits, instead of considering the marketing value of great training. People creating the CCNA certification and the CCNA Academy knew the importance of “infecting” people looking for generic networking knowledge with product-specific details like the configuration syntax. Once someone knows how to configure a particular device, they’ll usually recommend it when making a purchasing decision (because they’ll look like a wizard when it’s time to configure it). Apple knew that ages ago when they started offering student discounts on their laptops.
Documentation is no different. If you learn how Spanning Tree Protocol works from Cisco’s documentation, you might also learn how to configure it. Unfortunately, it takes at least an order of magnitude more effort to create excellent documentation than it takes to make an irrelevant step-by-step configuration task list followed by (oftentimes misleading or incomplete) command reference. If the documentation team cannot justify the value of excellent documentation to get a bigger budget3, they have no other choice but to participate in the industry-wide race to the bottom.
Now, imagine the vendor then makes it easy to test the concepts on their virtual devices. Most vendors got at least that particular message, with Cisco being a sad exception (apart from the Nexus OS image). It looks like the bureaucrats won.
Fortunately, there are still organizations producing decent documentation. It was easy to figure out ~90% of the time how things work from AWS or Azure documentation. Maybe it’s time we all move to the cloud 😜.
-
RSC – the 1-week course that told you everything you needed to know about Cisco devices in the mid-1990s. ↩︎
-
Obviously, I complained too much. I was never again invited to a similar event. ↩︎
-
The costs of the training and documentation teams are usually not even a rounding error in the vendor’s financials. ↩︎
A few reasons to name, perhaps, biased as well:
• Vendors and partners: "If the documentation is too good, we sell less tech support subscriptions and professional services.." • Training business: "If the courses are too good, people attend once and then share the knowledge horizontally.." • Customer tech gurus: "If the documentation and trainings are too good, my junior staff understands my awkward designs and configurations, can optimize them and threatens me out of my guru chair.." • General public: "We do not care, just give us a quick recipe that works to copy and paste.."
While a well-known author of a particularly obtuse RFC supposedly said, "It's not my job to educate the competition," I'm an optimist, hoping the above reasons (although all very much present) are still a minority view and that Hanlon's Razor applies to this case as well.
The way I see it, there are two main factors contributing to the decrease in quality for the documentation : - speed of change ; the change process is continuously accelerating ("new" features mostly) and people need to compromise on the details covered in the documentation, otherwise can't keep up with updating it - abstraction ; the underlying technologies (what is under the hood, how it works, etc.) are no longer interesting to the audience, as everyone wants an easy way of achieving something (if possible with a few clicks of a mouse)
While I agree with the "underlying technologies are no longer interesting to the wide audience", we're experiencing the same shift everywhere in IT, and yet Python or MariaDB (for example) still have pretty good documentation ;)
As for "speed of change," -- while it might be true in IT in general, I don't think it applies to networking, be it in the physical or virtual (VM or containers) world.
Just one data point (and I know that anecdotes != data), the VMware NSX documentation got worse and worse as the speed of change (= new features per release) decreased, which could imply the "we captured the addressable market, now it's time to cut costs" mentality.
Great documentation requires investment from the supplier,both Financial and personnel/ resource investment. This costs money and personnel resources are incredibly precious. Thus, the equation suppliers look at is what is the incremental revenue increase of producing great (versus mediocre) documentation and where else could I apply those precious resources (to new products or selling to big customers). Sadly, it is a financial decision and the numbers don’t clearly show the ROI. As a former engineer I can attest to the value of great documentation but right now feature velocity is prioritized. The good news is that if a supplier who produces great documentation starts winning in the market, others will follow. So vote with your dollars, include it in RFP selection criteria, and eliminate vendors with poor documentation from your shortlists…