Background

In 2013, I wrote a web services server for my friend’s popular photobooth software.  Until then, the photobooth software was a heavyweight client application.  However, he wanted to add more sharing and social functions, such as sending photobooth pictures to users via texting and e-mail.  Also, some recent changes in Tumblr’s API made uploading pictures directly from the client rather burdensome.   For these and other reasons, he wanted to create a web server to handle all of the social and sharing integration.

The Problem: So Many APIs to Connect Together

It seems that virtually every conceivable function is available via a REST API.  The primary purpose of the web services server is to act as “glue” for all of these various services.  

For this project, the initial set of APIs included:

• Cloud table storage for the server’s own data
• Cloud blob storage to temporarily store pictures
• Text messaging (SMS and MMS) service to send and receive text messages and photobooth pictures
• E-mail service to request and receive photobooth pictures
• Tumblr upload integration
• OAuth Callback API
• A photobooth client API to enable photobooth applications to access these web services

Some of these interfaces are outgoing-only, but others, such as the e-mail and SMS interfaces, are two-way and may receive calls from the cloud at anytime.  Additionally, we needed the ability to quickly add other social and sharing services to the server, support API versioning, and a server administration and testing interface.  Because these APIs are exposed to the web, user authentication, URL validation, and API parameter validation are extremely important.

Data-driven API Processing Eliminates the Tedium of Supporting Different APIs

Rather than trying to implement each of these APIs separately, I decided to build a data-driven, self-documenting, modular API processing system.  Each API is defined by a Javascript object that defines that API’s specific function calls, associated URLs, and parameters. This API specification object is used by the server to process all incoming web requests and generate a testing interface.

By controlling the API processing pipeline using the API specification object, changes or modifications to an API specification object are automatically implemented by the server. Additionally, new API functions and even entirely new APIs can be easily added just by passing their API specification objects to the API processing pipeline. Despite being code, the API specification objects are easy to read and understand.

API Specification Object

For example, this is an abridged API specification:

module.exports.PhotocacheAPIList = [
    {
        name:"uploadPhoto",
        version:"1",
        url:"api/cache/v1/uploadphoto",
        callback:"uploadPhotoToCache",
        enctype:"multipart/form-data",
        params: [
            {
                name: "uploadFile",
                type: "file",
                inputtype: "file",
                optional: true								
            }
            ,{
                name: "tempFilePath",
                type: "path",
                hidden: true,
                optional: false								
            }														
        ]
    } 
    , {
            name:"deletePhoto",
            version:"1",
            url:"api/cache/v1/deletecachedphoto",
            callback:"deleteCachedPhoto",
            params: [
                {
                    name: "cachecode",
                    type: "alpha-numeric",
                    optional:false
                }							
            ]
    }			       
];

This Javascript array includes objects representing each API function. Each API function’s specification object includes the function name, version, calling URL, the callback function used to process the API call, and any function parameters. For each API function’s parameters, this object specifies the parameter names, the parameters’ data types, and whether the parameters are required or optional.

Processing API Requests Using the API Specification Object

The web services server processes all incoming requests as follows:  

The server initializes the API processing pipeline with the API specification objects for the supported API. The API processing pipeline then uses these API specification objects to evaluate each incoming API request.

If all of the required parameters are present, then the server validates and sanitizes the API request’s parameters (including any optional or extraneous parameters). The server validates parameters using a standard validation module, wrapped in another module to allow for user-defined parameter types.

If the server matches an incoming API request to an API specification object and passes all of these validation steps, then it forwards the API request to the callback function in the API specification object for processing.

Other Uses for the API Specification Object

In addition to controlling the API processing pipeline, these API specification objects can be used for other purposes, such as for documentation and testing. For example, I also use it to generate a web-based testing interface. The server passes the set of API specification objects to a Jade web template and generates a web testing page including a set of web forms for the API functions. Each web form is configured with the URL and parameters of one of the API functions. The resulting web page includes web forms for all of the API functions.