Introducing Formplug v1, a form forwarding service for AWS Lambda

It’s estimated that approximately 269 billion emails are sent in a single day. Over 10 million were sent as you read that previous sentence. Even in 2018, over 40 years after its creation, email is still one of the most reliable and versatile means of communication on the internet.

Throughout this blog post I’ll be providing a technical walkthrough of the latest release of Formplug, an open-source form forwarding service that I’ve been working on. It makes it incredibly simple to receive form submissions by email. Typically you’d use Formplug in an environment where you don’t have the capability to execute sever-side code, like Github Pages for example.

View the project on GitHub

Try the demo

How it works

Upon deploying the Formplug service to AWS, you get back an API Gateway endpoint URL. This should set as the form action for any forms that you want to receive by email. Behaviour can then be customised using hidden form inputs prefixed by an underscore. For example, to set the recipient you would use the aptly named _to input.

Among other features, there’s support for encrypting email addresses, spam prevention and URL redirection. See the readme for the full range of configuration options.

Architecture

A Lambda in the context of AWS can be thought of as a function that gets invoked based on a predefined event.

Formplug is made up of two Lambdas:

A receive Lambda — for parsing form submissions.
A send Lambda — for building and sending emails.

In the case of the receive Lambda, events are generated from form submissions to our API Gateway endpoint (a public URL which is generated on the first deploy). In the case of the send Lambda the event is generated from an API call from the receive Lambda.

Configuration

Both of these Lambdas are defined in the serverless.yml configuration file, which describes the required AWS infrastructure. The infrastructure will be automatically instantiated on deployment using CloudFormation.

functions:receive:handler: src/receive/handler.handleevents:- http:path: /method: postrequest:parameters:querystrings:format: truesend:handler: src/send/handler.handle

For each Lambda we’re defining an exported JavaScript function as the handler. This will be invoked each time the Lambda’s event is triggered. The handler itself is just an arrow function which takes three arguments, the first of which is event which in our receive handler holds the form data itself.

module.exports.handle = (event, context, callback) => {

}

Whenever a form is posted to our endpoint, this function is invoked. In order to return a response we need to use the provided callback argument. This is an error-first callback so we should pass null as the first argument for a successfully formed response. The second argument should be an object that contains a status code and response body. The callback defaults to the application/json content type.

callback(null, {statusCode: 200, body: 'Email sent'})

Validating the form submission

At the top of the Formplug receive handler a new request instance is created and event is passed into the constructor. The Request class is responsible for validation. Defined in the Request constructor is everything that needs to be extracted from event. This is primarily just the recipients for the email and the form inputs to forward on.

class Request {constructor (event) {this.singleEmailFields = ['_to']this.delimeteredEmailFields = ['_cc', '_bcc', '_replyTo']this.recipients = {to: '',cc: [],bcc: [],replyTo: []}

this.responseFormat = 'html'  
this.redirectUrl = null

this.pathParameters = event.pathParameters || {}  
this.queryStringParameters = event.queryStringParameters || {}  
this.userParameters = querystring.parse(event.body)

}}

We don’t do any real work in the constructor. Also notice how we use the Node.js querystring module to set userParamters. This takes the request body containing the form data encoded as application/x-www-form-urlencoded and turns it into a JavaScript object.

Validate using promises

There’s a validate method on Request that’s responsible for parsing the event. It takes advantage of the ability to sequentially chain promises so that the resolution of one promise becomes the success handler argument of the next. Each success handler in the chained promise sequence is used to validate a different aspect of the form submission. Rejected promises bubble up to their top-level parent, so we can define a single catch method in the handler that’s shared amongst all of the validation methods.

validate () {return Promise.resolve().then(() => this._validateResponseFormat()).then(() => this._validateNoHoneyPot()).then(() => this._validateSingleEmails()).then(() => this._validateDelimiteredEmails()).then(() => this._validateToRecipient()).then(() => this._validateRedirect())}

If you’re wondering what one of these specific validation methods looks like, let’s look at _validateSingleEmails which is responsible for setting the _to recipient. It checks whether any singleEmailFields have been provided in userParameters, resolving the promise if the validation was successful and rejecting the promise if there were errors.

_validateSingleEmails () {return new Promise((resolve, reject) => {this.singleEmailFields.filter((field) => field in this.userParameters).forEach((field) => {let input = this.userParameters[field]if (!this._parseEmail(input, field)) {let msg = `Invalid email in '${field}' field`let err = new HttpError().unprocessableEntity(msg)return reject(err)}})

return resolve()

})}

HTTP errors are rejected

Instead of rejecting a JavaScript error directly in the promise, an error is generated from a HttpError method. This class has been created to provide a friendly API for generating common HTTP response errors. This class contains a public method for each supported error response. Shown below is an example of the class with just the 422 unprocessable entity response.

class HttpError {unprocessableEntity (message) {return this._buildError(422, message)}

_buildError (statusCode, message) {const error = new Error(message)error.statusCode = statusCodereturn error}}

You might be thinking that it’s not worth structuring error handling like this as requires more upfront work. The benefits become more apparent when you consider how clean the receive handler can become. We can reject a HttpError error from anywhere in the promise chain and have it caught in the top-level catch method. At this point an appropriate response can then be built for display to the user.

Generating responses

To generate a response it’s not quite as simple as providing a status code and message to callback. Both JSON and HTML content types are supported, each require different headers to be set and have a different body format. Redirect responses through the _redirect form input are also supported, which similarly has different requirements.

A class has been created that can build the different response objects that can be passed to callback.

class Response {constructor (statusCode, message) {this.statusCode = statusCodethis.message = message}

buildJson () {return {statusCode: this.statusCode,headers: {'Access-Control-Allow-Origin': '*','Content-Type': 'application/json'},body: JSON.stringify({statusCode: this.statusCode,message: this.message})}}

buildHtml (template) {return {statusCode: this.statusCode,headers: {'Content-Type': 'text/html'},body: template.replace('{{ message }}', this.message)}}

buildRedirect (redirectUrl) {return {statusCode: this.statusCode,headers: {'Content-Type': 'text/plain','Location': redirectUrl},body: this.message}}}

To make use of the above class you would first create an instance of it by passing in a status code and message, as this is common across all response types. You would then call one of the three public methods according to the desired response.

For example, to build a JSON a response you would do the following.

const statusCode = 200const message = 'Form submission successfully made'const response = new Response(statusCode, message)callback(null, response.buildJson())

By default a HTML response is shown by with a generic success message.

This page is generated by loading a local HTML template file and doing a find and replace on a {{ message }} variable. The template is loaded in the handler using the Node.js fs module.

const path = path.resolve(__dirname, 'template.html')const template = fs.readFileSync().toString()

const message = 'Form submission successfully made'const html = template.replace('{{ message }}', message)

Determining the response type

The Response class is responsible for building the responses, but it has no concept of what the appropriate response is. The choice of the response is determined by a series of condition statements that look at the validated request created earlier in the receive handler.

if (request.redirectUrl) {callback(null, response.buildRedirect(request.redirectUrl))return}if (request.responseFormat === 'json') {callback(null, response.buildJson())return}if (request.responseFormat === 'html') {const path = path.resolve(__dirname, 'template.html')const template = fs.readFileSync(path).toString()callback(null, response.buildHtml(template))return}

If you weren’t aware, you can return undefined in JavaScript by not specifying a return value. In the code block above we’re using it to stop the script execution so that callback is only ever invoked once.

Structuring the receive handler

Building again on the ability to sequentially chain JavaScript promises, the receive handler takes the following format.

module.exports.handle = (event, context, callback) => {const request = new Request(event)

request.validate().then(function () {// send email}).then(function () {// build success response}).catch(function (error) {// build error response}).then(function (response) {// response callback})}

Objects built using Response are passed to callback to return a HTTP response in the final success handler on this top-level promise chain. This response object could have been resolved from either the previous then() or catch() method. Replacing the above comments with implementation code we arrive at the finalised handler.

module.exports.handle = (event, context, callback) => {const request = new Request(event)request.validate().then(function () {const payload = {recipients: request.recipients,userParameters: request.userParameters}return aws.invokeLambda('formplug', 'dev', 'send', payload)}).then(function () {const statusCode = request.redirectUrl ? 302 : 200const message = 'Form submission successfully made'const respnse = new Response(statusCode, message)return Promise.resolve(response)}).catch(function (error) {const response = new Response(error.statusCode, error.message)return Promise.resolve(response)}).then(function (response) {if (request.redirectUrl) {callback(null, response.buildRedirect(request.redirectUrl))return}if (request.responseFormat === 'json') {callback(null, response.buildJson())return}if (request.responseFormat === 'html') {const path = path.resolve(__dirname, 'template.html')const template = fs.readFileSync(path).toString()callback(null, response.buildHtml(template))return}})}

Invoking the send Lambda

Not discussed is how emails are actually sent. Emails get sent using Amazon Simple Email Service (SES) in the send Lambda, which is invoked in the first promise success handler in the receive Lambda. It’s invoked with a payload containing the recipients and form inputs.

const payload = {recipients: request.recipients,userParameters: request.userParameters}

return aws.invokeLambda('formplug', 'dev', 'send', payload)

The Lambda is invoked using the AWS SDK available on NPM. Although it might look as though we’re directly calling the AWS SDK, we’re not. We’re actually calling a method on a singleton instance of a wrapping class. It provides a proxy to the library, exposing only the relevant methods in a simpler API.

const aws = require('aws-sdk')

class AwsService {constructor (aws) {this.aws = aws}

invokeLambda (serviceName, stage, functionName, payload) {let event = {FunctionName: `${serviceName}-${stage}-${functionName}`,InvocationType: 'Event',Payload: JSON.stringify(payload)}return new this.aws.Lambda().invoke(event).promise()}

sendEmail (email) {return new this.aws.SES().sendEmail(email).promise()}}

module.exports = new AwsService(aws)

Sending emails

The send Lambda first creates an instance of Email and then calls the sendEmail method on the previously described AwsService class. In this handler, the event argument is just the payload from the receive Lambda.

module.exports.handle = (event, context, callback) => {const email = new Email(config.SENDER_ARN, config.MSG_SUBJECT)email.build(event.recipients, event.userParameters).then(function (email) {return aws.sendEmail(email)}).catch(function (error) {callback(error)})}

Config variables are loaded from a local JSON file. The SENDER_ARN variable is the Amazon Resource Name of the sending email address (SES only sends emails from verified email addresses).

const config = require('./config.json')

The build method on the Email class creates an SES compatible object to pass to to the AWS SDK. It first checks that SENDER_ARN is valid and then returns the SES object.

build (recipients, userParameters) {return this._validateArn().then(() => {let email = {Source: this._buildSenderSource(),ReplyToAddresses: recipients.replyTo,Destination: {ToAddresses: [recipients.to],CcAddresses: recipients.cc,BccAddresses: recipients.bcc},Message: {Subject: {Data: this.subject},Body: {Text: {Data: this._buildMessage(userParameters)}}}}

  return Promise.resolve(email)  
})

}

The email body is built by looping over the user parameters that were sent as part of event. Formplug configuration variables that are prefixed by an underscore are omitted.

_buildMessage (userParameters) {return Object.keys(userParameters).filter(function (param) {// don't send private variablesreturn param.substring(0, 1) !== '_'}).reduce(function (message, param) {// uppercase the field names and add each parameter valuemessage += param.toUpperCase()message += ': 'message += userParameters[param]message += '\r\n'return message}, '')}

Wrapping up

I hope you found this high-level codebase walkthrough interesting. For more information and for deployment instructions, you should go check out the repository on Github. If you have any comments or suggestions, please leave a reply below and I’ll happily get back to you.

View the project on GitHub

Try the demo

danielireson/formplug-serverless_formplug-serverless - Form forwarding service for AWS Lambda_github.com

If you enjoyed this blog post you might also enjoy a previous medium tutorial where I built a serverless URL shortener using AWS Lambda and S3.

How to build a Serverless URL shortener using AWS Lambda and S3_Throughout this post we’ll be building a serverless URL shortener using Amazon Web Services (AWS) Lambda and S3. Whilst…_medium.freecodecamp.org