What is Serverless?

No Servers? Nope. There are still servers. Serverless is simply a new paradigm of computing that abstracts away the complexity associated with managing servers. As a developer you will be focused writing application logic without worrying about server configurations or operations. They are completely taken care by the cloud provider. You write and deploy the code as a Function, then start consuming it. As simple as that. No need to start instances or choose operating systems. Also no need to worry about load balancing or downtimes. They are taken care of in Serverless by the cloud provider.

Why Serverless?

Low Cost: In Serverless, there are no running instances. Code/Function will run only when it is called. So, you will be charged only for the resources it consumes when it’s executed. Idle time is not charged. This is one of the main reason for going Serverless.

Nearly Zero Administration: As a developer, you need not take care of software updates and downtimes etc.. They are taken care of in Serverless by the cloud provider.

Scalable: Traditionally to keep an application scalable for the incoming traffic, multiple instance of the app must be launched and a load balancer is required to route the traffic flow. Also, its really difficult to handle the sudden traffic fluctuations. Launching more instance might take time. Where as with Serverless its totally handled by the provider. Any number of function instances could be launched to serve the user traffic.

Developer Productivity: As we need not worry about servers, we could spend time on actual development of the app. It helps focusing your efforts on what provides value to the users.

Though going Serverless is advantageous compared to traditional way of running instances, it may not be the option for all types of applications. Some applications demands more control over the server configurations, caching, state management etc.. where Serverless isn’t the option.

Getting Started

Let’s understand by setting up a Serverless service. For the scope of the article we will be using AWS Lambda and API Gateway to create the service with Node.js as runtime.

The default template contains two files serverless.yml and handler.js. yml file contains the configuration of the service such as provider, runtime, functions, http events etc..

This basic configuration when it is deployed, creates a service demo with a function hello and /hello/test http endpoint for GET requests.

handler.js exports the hello function with the simple definition as follows.

We are ready to deploy the service.

The service could be deployment gives more control such as deployments to a specific region and to environments such as devtestuatprod etc..

And We Are Done !! The service is now available to the world with the following URL pattern.

Every time the URL is hit, the function will be invoked and send the response.

So, Are we all Good to go with Serverless for all apps?

Maybe not always. Talking about Serverless, the common problem we hear is a Cold Start. Sometimes it becomes the decision point to choose Serverless architecture for the app. Let us understand what it is and how can we solve it.

The Cold Start Problem

When a function is invoked for the first time, it has to load in some servers in the cloud before running as there won’t be any running instances. So, the total time it takes = (time to load) + (time to run). Because of this startup function takes the longer time to run for the first time. This phenomenon is referred as a Cold Start. Any request that is made later, doesn’t require a startup as the function is already loaded and is ready to run.

                                                                             Function Lifecycle

So is the Cold Start is only for the first request? If so, why would it be an issue? Next thousands of requests would be faster. Right? Let’s find the answer by performing some tests for different use cases.

Note: And all values represent total time a request took in milliseconds. And the tool used to perform these tests is available here.

Test 1: Concurrency

                                                Concurrency 1, memory: 1024MB, App size: < 1KB
                                            Concurrency: 5, memory: 1024MB, App size: < 1KB

This clarifies that every first concurrent request is a Cold Start. So, in short, Cold Start is not for the first invocation of your function, it happens for each first concurrent invocation.

Test 2: Function memory (memory required to run)

                                           Concurrency: 1, memory: 3008MB, App size: <1KB

Allocating more memory improved the results. Cold Start is significantly better for functions with more memory.

Test 3: Code size

                                              Concurrency: 1, memory: 1024MB, App size: 16MB

Function with more code took slightly more Cold Start time than smaller one.

Note: Large npm packages were imported to increase the bundle size of the app, but they were not used. Function was still the same as above tests.

Test 4: Different Runtimes

                                             RunTime: Java 8, Concurrency: 1, Memory: 1024MB
                                            RunTime: Java 8, Concurrency: 5, Memory: 1024MB

Every Runtime has different Cold Starts. Runtimes like Node.js, Go performed much better than Java with respect to Cold Start. (I am not biased towards any Runtime by the way 😄).

How to avoid?

It really depends on the type of app. For applications like a user portfolio, a Cold Start isn’t really problem. Where as for applications like e-commerce with large users its really important to serve the request as fast as possible.

Keep the functions warm: There is no Cold Start for the functions which are warm. So, keep the functions warm with the help of services such as AWS CloudWatch. As we understood that a Cold Start happens for every concurrent request, we need to understand how many instances needs to be warm. For an example, lets say some sale goes at 2 PM and expected traffic would be 10,000 concurrent requests, those many function instances could be pre-warmed couple of minutes prior to the sale.

Choose appropriate Memory: Functions with more memory had better Cold Start than the ones with less memory. However, by allocating more memory increases Cost. Choose an ideal memory.

Keep the code optimal: Function startup with bunch of code would be slow. Its always important to keep the code size optimal. Remove unnecessary dependencies.

Choose Appropriate Runtime: As we could observe every runtime has different results, choosing a runtime plays a role in Cold Start.

Go Serverless !! 🤘

Demos used in the article are available at: https://github.com/madhusudhand/serverless-demo

Github: https://github.com/madhusudhand

Twitter: https://twitter.com/madhudollu