Brain Damage : Diagnosis & Treatment

Using software libraries is an important skill, as any professional software developer will attest. However, if software libraries are introduced too soon, they end up forming a wall of abstraction that leave smart people dumbfounded. Here my friend was, thinking she was on the cusp of learning the secret rules that govern computers; but instead, the instructor simply swapped out monolith of user-facing software with the equally opaque abstraction of the client library. Without mastering the language first — and learning how to read the library source code — my friend didn’t have the tools she needed to dig deeper and figure out what was going on at the bottom of the call chain.

from www.evanmiller.org/you-cant-dig-upwards.html

The recommendation here is to learn C and move upwards to Python and maybe Lisp. 
There have been a few popular posts about Java not being a good starting point 

• Perils of Java schools by  Joel Spolsky  
•  Computer Science Education:Where Are the Software Engineers of Tomorrow?  by Dr. Robert B.K. Dewar and Dr. Edmond Schonberg   
• Execution in the Kingdom of nouns by Steve Yegge

Here's a nice example - let's take an array of words like "death_star" and return an array of words like "Death Star", in Javascript, you _can_ do this :

--------------------------------------------------------------------------------------------------------------------
a.split('_').map( function(x) {return x.charAt(0).toUpperCase() + x.slice(1); } ).join(' ');
--------------------------------------------------------------------------------------------------------------------

If your company allows you to use Google Guava, then you can do this in Java:
--------------------------------------------------------------------------------------------------------------------
Function capitalizer = new Function<String,String>(){
                        @Nullable
                        @Override
                        public String apply(String s) {
                            return s.substring(0, 1).toUpperCase() + s.substring(1);
                        }

                    };

Joiner.on(" ") .join( Lists.transform( Splitter.onPattern("_").splitToList(key), capitalizer).toArray(new String[]{}));
--------------------------------------------------------------------------------------------------------------------

Now, one of them's much more readable than the other. And it's not the second one.
Also, the Kingdom of Nouns requires an appointed Noun Officer to be present for every verbing. Hence the "Joiner" and "Splitter" thinggies. Also note the short and succint function creation (Not!!)
Of course, Guava is what the cool kids in Java would use, if by cool kids you mean middle aged balding uncles wearing tee-shirts with collars for whom the words "Hitchhiker's guide to the galaxy" mean nothing.
Some programmers would use Apache's StringUtils, but these used to work in Sun Microsystems, and have now been assimilated by the Oracle machine and had their organs harvested.


And there's the more pedigreed lament by the creator of Shen  and a professor of computer science - Mark Tarver - marktarver.com/professor.html

All these point to what Alan Kay said all along , to paraphrase - any culture that spreads faster than education is a pop culture.  He also said these two following nuggets - from the Wikiquote page -

• Sun Microsystems had the right people to make Java into a first-class language, and I believe it was the Sun marketing people who rushed the thing out before it should have gotten out.
• If the pros at Sun had had a chance to fix Java, the world would be a much more pleasant place. This is not secret knowledge. It’s just secret to this pop culture.

So, how to avoid years of Java brain damage - In this case 16 ? 
A parting quote from John Carmack - 
"Low-level programming is good for the programmer's soul."

Still not convinced ? Or you _ARE_ convinced and would like to flog that dead horse some more ?Here's one more of my favorite bloggers who won my admiration by coming up with 
"Organizational Skills Beat Algorithmic Wizardry" , only he's talking about OOP and not Java

Don't Distract New Programmers with OOP
OOP Isn't a Fundamental Particle of Computing
Moving Beyond the OOP Obsession

I'll leave you with this list of dignitaries laying it on thick - if  you're still not convinved, you're in IT  , and not Engineering. The robots will be coming for your job soon

http://www.yegor256.com/2016/08/15/what-is-wrong-object-oriented-programming.html


The END (?)

What a micro-service isn't - part 1

A micro-service is usually a web service . All web services aren't micro-services. To get promoted, you need to do a lot more.
Once you add caching, load balancing, failover, integrate logging aggregation and follow it with metrics and dashboards and alerts, then publish documentation on the intranet portal, you're probably good to go.
Your teensy-weensy web service is now a grown-up ,card-carrying member of society of services that make up your organization.
The current poison of choice is REST, mostly because it's simple - never underestimate simple - and open source - never underestimate availability of developers/coders  as  factor in setting the culture of a field.

Let's take each one by one.

• Caching

There are multiple types of this. Usual suspects are Varnish on the outside and one of Ehcache, Memcached, Redis, Hazelcast etc  on the inside for application cache. Teracotta has gained an advantage by going one step further and providing easy-to-configure Hibernate level 2 caching so that Java developers need to only wag their fingers very little to get it going.

• Load balancing & Failover

I'm ashamed to admit that I don't know what software provides load balancing. I assume nginx does, but I don't know. Some people roll their own.
Failover is easy - you make exactly one more copy of your web -service. Or better still,  install the same thing on another app server, preferrably on another hardware server, preferrably one more in another colo.
You'll also need a reverse proxy somewhere in the mix.

• Logging aggregation & metrics

Hadoop for the win !!! we can now aggregate ALL your logs for 15 days for $$$$ and give you a unified browser based search interface.
No more SSHing into remote servers in the midlde of the night and doing a tail -f .
You just have to remember the proprietory (TM) query syntax and hope you don't need to use a regex to extract some field from your log statements to figure out what's going on when paged.
Search for "SPLUNK" on the interwebs.

• Metrics and dashboards and alerts

See above.  Also this. Nagios and ChekMK are like plumbers. Grungy, dirty but cannot be done without. They provide metrics and graphs . So does New Relic. All of them run agents on your hardware.

• Documentation

Swagger. That's all I'm going to say about this.

And once you relax, knowing that you're good to go, remember you aren't.

There's Security to worry about. OpenID or OAuth . And tokens. And user directories.
And domain names and  SSL certificates. And a mail server / service.

And your org ALSO needs 

• CI (Continuous Integration)

Usually Jenkins integrated with your Github which runs your JUnits and then deploys.

Usually. 

It could also be Jenkins integrated with your SVN which runs your TestNGs.

Or  Microsoft Team Foundation Server and deploy by hand. 

More often than not, CI is only C till the compilation part. 

Chickening out before the deploy, or keeping the 'deploy' part for a dramatic hi-priority conference call/remote login session  at midnight is pretty common.

• Automated Deploy !

There's Chef, Puppet, Capistrano. shell scripts.  A $$$$$ licensed version of IBM Urban Deploy (which just runs your shell scripts).

• A/B testing & Experiments

Ideas come from statistics. Finch .

• Analytics 

See next.

• Hadoop for the Marketing team

We don't know what they do with this, but apparently we can use computers to keep the room warm against the terribly powerful air-conditioning by counting words in files three times each.
 And Informatica licenses. Also SAS licenses . And maybe a Revo R license because SAS doesn't do arrays or some such.
Squiggly colored lines on a white checked background are the key to unlocking budgets.

Care and Feeding of your Developers -

 what you need 

• reddit and imgur for distractions and knowledge, hacker-news if you have hired Intellectuals 
• youtube for music while working
• an O'reilly Safari account  
• coffee machine & free snacks
• IntelliJ helps
• one Hipster rails developer to show others what cool looks like

Observant reader that you are you will now notice that the heading is about what a micro-service isn't, and the blog post is all about what it is .  

It is in such paradox that sanity thrives and flourishes in this beloved industry of ours, and lets us use terms like agile-waterfall with a straight face, where sprints scheduled one after the other technically constitute a marathon which , as life coaches advice us  - should not be sprinted.

Let me leave you with something more solid than pop-philosophical ramblings.

A micro-service is not a UI. So budget for that too.

Microservices - State of the Art ( Clue - it's ripe for Bangalore-ing)

All this stuff is stuff gleaned off Goto conf videos (  go to/see them,they're good  !! )

This first one's from Kevin Goldsmith talking about Spotify. Youtube link here
Among other things, it is mentioned that unlike typical orgs having a Dev team, a QA team, a deploy team, an infra team etc, the micro-servicey organization has all these functions served by a single team, which is responsible for a (preferably) single business function. That means the, say, Users team handles development, but also deploys to its own servers and does its own QA.
Point to note is that this is an American way of doing things -as in this makes a lot of sense to a growing business. A director in a large company might not want to have the many many teams under him dissolved, so large companies will continue to have multiple teams, or even roles until forced to, much the same way large companies did not do away with the waterfall model until they HAD HAD to, as in, all those doing "Agile-Waterfall" were fired and had to go to separate separate companies where no one practised "Agile-Waterfall" ,and they were forced to learn "Agile", which we wish they didn't.

  
  
Kevin G also talks about what a typical micro-servicey company can do. See below. We are no longer slow-moving single-focus behemoth with multiple layers of fat . We are a lean mean Rambo, keep give us something to shoot. We here being the Company, of course.


Senior BeardFace shows us what's happening in today's world of hardware


That's Fred George above, talking about challenges. Youtube link here
It's hard to take someone with two first names seriously, but Michael Lewis wrote a great book (Liar's Poker) , so I'll bite.
Remember how sometime when you had requested IT for a new server around the time Steve Jobs died, and last week your manager told you that we were going in a "different direction" - meaning you're not gonna get that server, and you stopped dreaming about increasing conversions/ traffic to your service and started leaving early to binge-watch Netflix ?
Your micro-servicey org is dead in the water if you don't trust your engineers to ask for and get servers at will. Containers like Docker ensure that the version of OS they use never gets auto-upgraded  to something that doesn't work, unlike the frequent sneak upgrades your office Windows or Mac laptops inflicts on you.
There's no protection against Docker versions not working with other Docker version though, except for diligence on the Dockerer's part .

Senior BeardFace also knows how quickly teams are delivering code these days
Or wishes they did. Maybe they do in California. Please don't show this to my manager.


Scientist BaldyBeard  compares and contrasts the Olde and New Worlds
This is a long and very informative talk on Microservices. Like much of this man's work, indispensable if you want to work in the industry. Youtube link here
Slide below points out the main features of one vs the other.
What is not seen here is that microservicey orgs have large jumbles of github repositories and many many services, or many versions of the same services , in various stages of completeness . Tread lightly if you don't trust your engineers.
The primacy of engineers is integral to a micro-servicey org, lacking which you'll quickly find yourself gravitating towards an IT-ish org following the flight of engineers, and then all you'll see is a large number of releases involving config-file changes until the project is replaced by newer one.
This is perfect for an offshore cost center, as opposed to an onsite profit center, hence the title.




Scientist BaldyBeard has a list of attributes by which the MicroService may be known
Again, an absolutely indispensable list.  Points 1-4 means small teams serving a business function with code. Let's not get diverted by the kind of conflicted thinking that comes with projects.
See this link for some grade A enlightenment w.regards to the manager-project-sociopathy
Point 3 asks to align business goals with jobs, and virtually no one will tell you that their business is not aligned to jobs. Gee
An automated deployment pipeline is invaluable, and helps bring team sizes down. So this will work only at a profit center ,remember.


After Amazon,  Business has changed forever. It's hard to imagine one kiloton of Java code showing you wares ,scanning warehouses,  taking your money , routing deliveries to you etc
but that's  the nameless faceless labor force of the future (which is actually today) .
And what Java can do on some non-unique electronics, it can do on some other.
Like Fire which needed to be  brought down from the Gods but spread like wildfire afterwards,
high-volume massively-scalable systems are changing everything , and their magic  will be available to everyone at the cost of a code monkey and a Docker container.
While PHP wrought the first true magic of e-commerce , Java will create centralized players absorbing the business of millions across the globe [1] (in a world of 7 billion, I assume only some millions have _money_) . Commodity hardware, commodity software and commodity programmers will make the technology ripe for cost-optimization [2] ( hello Bangalore !) where your flimsy bug-ridden codebase can be kept on life support for a hundred years by means of a highly tantric ritual of daily meetings and scheduled releases of one line changes in order to appease the Code Gods. And as the software arteries harden , business will move on to other sparkly new things.[3][4]

In part 2 of this series , I'll talk about what makes microservices different from a small hello-world Java program that you might write on your first day of Software ( answer - not much ).

[1]  Cue emergence of  a giant node 
[2]As the language of microservices become household terms, the pop culture will take over  -  the technology will spread faster than education - Java will see to that - and the system will stratify and harden - MBAs  will swoop in and all existing code will get shipped to Bangalore.
[3] Microservices will remain alive in Chicago, but I doubt they'll survive in the Valley.
 [4] It's time someone created a code-aware self-pruning , genetic-programmingy L-system of enterprise code.