Will Tensorflow Serving Ease Data Science Operational Pain?

One of the more exciting things I heard lately was Google continuing to open source more and more of the TensorFlow ecosystem with the release of TensorFlow Serving. 

TensorFlow
Http://www.tensorflow.org

From their site… TensorFlow™ is an open source software library for numerical computation using data flow graphs.

TensorFlow Serving
https://tensorflow.github.io/serving/

From their site… TensorFlow Serving is a flexible, high-performance serving system for machine learning models, designed for production environments. TensorFlow Serving makes it easy to deploy new algorithms and experiments, while keeping the same server architecture and APIs. TensorFlow Serving provides out-of-the-box integration with TensorFlow models, but can be easily extended to serve other types of models and data.

It is exciting news to see these types of releases and advances. The operationalization of data science is very challenging for any organization. Managing model builds, versions, deployment and maintenance is extremely challenging technically and procedurally within the organization. These products go part of the way to helping improve this state of affairs and are a welcome addition to the data science tool chain.

Level of Effort

When you try to estimate and then track how much time a software engineering project and individual tasks within it will take. You need an agreement for that on your team. For years I have used something I made up when I worked at an Agency because story points didn't work well for my needs. Later, that was revised by the engineering team at Ekho to be easier to understand and use. I thought the method might be useful to others so I'm posting it here. This is nice for estimations and simple, since you can just take anything > 10 and divide by 10 to get number of sprints. This can be translated easily to calendars, gantt charts, etc.

Each request that flowed into our issue tracking system and got assigned to a sprint was assigned an LOE when it was accepted or estimated. This gave the team the ability to gauge if they were on track as well and make reasonable promises about due dates. I would generally say that once your team is good at using this scale to estimate you can reliability predict the data of a project by +/- 5% and that isn't easy.

Any single feature or job that scores in that high range of 8-10+ really should be carefully analyzed before assignment and every attempt made to break it down into smaller pieces. Sometimes, to do big things, there will be big multi-sprint LOE's. But, that should be rare in most software development projects.

Level of Effort (LOE) SCALE

This scale assumes one week sprints. It's easy to adapt to different length sprint schedules.

LOE 1 = 1/2 day for one Full Time Employee (FTE)

0 - A quick fix

1 - Some work that will take about one half(1/2) day for one FTE

2 - Some work that will take about one day for one FTE

4 - Some work that will take between about 1/2 sprint for one FTE (about 2-3 days)

8 - Some work that will take a little longer than 1/2 sprint (around 3-4 days)

10 - Some work that will take 1 full sprint (i.e. 1 work week)

10+ anything over 1 sprint (1 week) (i.e LOE 20 = 2 weeks, LOE 30 = 3 weeks)

Singularity University GSP Application Process

GSP 2016 Applications are now open. This is a truly life changing program for the participants and ultimately all the people they will impact down the line.

Would you like to make a 10ˆ9 impact on humanity? Could you positively impact the lives of one BILLION human beings in only 10 years or less? Yes, you can.

Applications are open so visit the site and check out the information there if you think you want to leverage Exponential Technology to make a 10ˆ9 impact on humanity.

Global Solutions Program 2016

Load Testing

Load Testing is multi-disciplinary. It's just one aspect of the broader concept of scalability of any software application. It requires broad experience to get it right. Done right it will net out several projects that will improve the overall scalability of the site / product in question in several dimensions best illustrated by the AKF partners scale cube. 

There are many factors that impact load testing on a mature application...

• server configuration
• network
• data center setup
• single points of failure
• dev/stage/prod environment separation
• database... oh the databases...
• the CDN
• the graphics coming out of the design team
• Mobile network speeds and latency 
• Inefficient Code / Bugs that only come out under high load or concurrency
• the deployment process
• missing documentation
• botched parameters on service / micro-service configs
• and this is the short list...

Meaningful load testing considers all these things and more. Done right, it shines a light on people and processes that are sometimes surprising. It can be painful because it is DESIGNED to show the flaws and break people's hard work. As a result successfully spawning projects from load testing results takes a bit of finesse and often executive support. It's very much about overall operations and site building process. Knowing the ins and outs of the entire product development lifecycle is a bonus. Serious load testing falls into the CTO / SR. Exec Tech role. It is not particularly fun work most of the time. This is why everyone puts it off for as long as possible.  

Oh Thoreau

I went to the woods because I wished to live deliberately, to front only the essential facts of life, and see if I could not learn what it had to teach, and not, when I came to die, discover that I had not lived. I did not wish to live what was not life, living is so dear; nor did I wish to practice resignation, unless it was quite necessary. I wanted to live deep and suck out all the marrow of life, to live so sturdily and Spartan-like as to put to rout all that was not life, to cut a broad swath and shave close, to drive life into a corner, and reduce it to its lowest terms, and, if it proved to be mean, why then to get the whole and genuine meanness of it, and publish its meanness to the world; or if it were sublime, to know it by experience, and be able to give a true account of it in my next excursion. — Henry David Thoreau[4]