Step-by-step guide to deploy a Meteor Application to AWS Elastic Beanstalk in Production

Published in

416serg’s blog

9 min readJan 26, 2017

It’s been a long time coming. Like long… and annoying… and frustrating… but we made it happen! It might be a long read, a lot of it will be code, I wanted to put everything here to serve as a one-stop-shop for anyone who wants to achieve the above.

I’ve been developing with Meteor for over 2 years now and it’s still my go-to choice for a project/app of any sorts. I’ve built a Slack app with Meteor (which I have now taken down for the time being cause it was on Modulus and I haven’t had time to transfer it over), a platform for people without degrees to learn more about career paths that don’t require them, and an HR solution to make your job postings stand out.

All-in-all it’s been a pretty smooth ride in terms of maintaining and keeping the sites up and running. I’ve tried Modulus (now Xervo), Digital Ocean (with mup), and of course, Galaxy. I’ve got nothing but great things to say about Galaxy:

Super easy to set up
Super easy to deploy
Great support
SSL comes with it
Autoscaling and such

But being part of a startup, an early startup, I must keep in mind the spendings and making sure that the dev stack is as optimized as possible. Having a development and a production environment on Galaxy ran us $60/mo for 2 separate containers.

AWS was really generous by allowing us to join their Activate Program and offering credits for their services and support. Without any prior experience of setting up servers on AWS, it was a challenge to execute this switch. After countless support threads and other articles, I am going to walking through it step by step. This is the way I got it to work, it’s probably not perfect, but it’s doing the job as of right now and I am always open to comments/suggestions so if you have any, comment away!

Hopefully this one article will be enough for someone to get going as I will try to write it for myself, if this is all I had to go off of.

Prerequisites:

METEOR@1.4.2.3
MongoDB hosted somewhere (I have it on Compose)
AWS Elastic Beanstalk
AWS EB CLI
Terminal
SSL Certificate

Meteor Setup

This one is probably not as optimized as it can be, so I am always open to suggestions here. The biggest challenge was Meteor.settings since Elastic Beanstalk env.process variables do not support storing JSON in environment variables. This is because the environment variables are stored as key/value strings in unencoded JSON (apparently, in CloudFormation).

So here’s what I did:

Inside imports/startup/client I created a beanstalk-fix.js file

import { Meteor } from 'meteor/meteor';if (Meteor.isProduction) {
  // public settings that need to be exposed to the client can be added here
  var publicEnvs = {
    /* Your Settings from Meteor.public */  
  };
  Meteor.settings.public = publicEnvs;
  __meteor_runtime_config__.PUBLIC_SETTINGS = Meteor.settings.public;
}

Inside imports/startup/server I created a beanstalk-fix.js file

import { Meteor } from 'meteor/meteor';if (Meteor.isProduction) {
  var meteorFile = process.env.METEOR_SETTINGS_FILE;
  if(meteorFile == undefined) throw new Error(
    'METEOR_SETTINGS_FILE env variable must be defined in production.')
  var fs = Npm.require('fs');
  var pjsonBuf = fs.readFileSync( meteorFile );
  Meteor.settings = JSON.parse( pjsonBuf.toString().trim());
}

More on the METEOR_SETTINGS_FILE later.

Now you can do

npm install --production
meteor build --directory ../build/ --architecture os.linux.x86_64

This will create a build folder outside of your app so it doesn’t cause any conflicts so you should have

directory
  app
    Meteor stuff  build
    bundle
      The Node app

AWS Stuff

Before deploying to EB, you need to create a few config files that go inside a special folder called .ebextensions. You use these files to specify the configs for the EB environment.

Application Load Balancer

Elastic Beanstalk used to come with Elastic Load Balancing, which, as far as I can tell, didn’t work nice with Meteor. It didn’t have Session Stickiness and it was hard to configure DDP. They recently came out with Application Load Balancer that has Websocket support and Session Stickiness out of the box. Here are the config files you’ll need to put into .ebextensions

alb-default-process.config

option_settings:
  aws:elasticbeanstalk:environment:process:default:
    DeregistrationDelay: '20'
    HealthCheckInterval: '15'
    HealthCheckPath: /
    HealthCheckTimeout: '5'
    HealthyThresholdCount: '3'
    UnhealthyThresholdCount: '5'
    MatcherHTTPCode: null
    Port: '80'
    Protocol: HTTP
    StickinessEnabled: 'true'
    StickinessLBCookieDuration: '43200'

alb-secure-listener.config

option_settings:
  aws:elbv2:listener:443:
    DefaultProcess: default
    ListenerEnabled: 'true'
    Protocol: HTTPS
    SSLCertificateArns: your SSL certificate ARN

More on SSL with Application Load Balancer here.

alb.config

option_settings:
  aws:elasticbeanstalk:environment:
    LoadBalancerType: application

That creates an Application Load Balancer as opposed to Classic Load Balancer.

Nginx

Here are some configurations to make sure you’re using nginx with the application, enable SSL redirects in the proxy and enable websockets.

nginxoptimization.config

files:
  "/opt/elasticbeanstalk/#etc#nginx#optimized-nginx.conf":
    mode: "000644"
    owner: root
    group: root
    encoding: plain
    content: |
      # Elastic Beanstalk Managed# Elastic Beanstalk managed configuration file
      # Some configuration of nginx can be by placing files in /etc/nginx/conf.d
      # using Configuration Files.
      # http://docs.amazonwebservices.com/elasticbeanstalk/latest/dg/customize-containers.html
      #
      # Modifications of nginx.conf can be performed using container_commands to modify the staged version
      # located in /tmp/deployment/config/etc#nginx#nginx.conf# THIS FILE HAS BEEN MODIFIED BY AN `.ebextensions` SCRIPTuser  nginx;
      worker_processes  1;error_log  /var/log/nginx/error.log;pid        /var/run/nginx.pid;worker_rlimit_nofile 65536;events {
          worker_connections  8192;
          use epoll;
      }http {client_max_body_size 100m;
        port_in_redirect off;
        include       /etc/nginx/mime.types;
        default_type  application/octet-stream;log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                          '$status $body_bytes_sent "$http_referer" '
                          '"$http_user_agent" "$http_x_forwarded_for" '
                          '$request_time $upstream_response_time';access_log  /var/log/nginx/access.log  main;sendfile        on;keepalive_timeout  65;# Elastic Beanstalk Modification(EB_INCLUDE)
        log_format healthd '$msec"$uri"'
                           '$status"$request_time"$upstream_response_time"'
                           '$http_x_forwarded_for';include /etc/nginx/conf.d/*.conf;
        # End Modification
      }
  "/etc/security/limits.conf":
    mode: "000644"
    encoding: plain
    owner: root
    group: root
    content: |
      # THIS FILE HAS BEEN MODIFIED BY AN `.ebextensions` SCRIPT
      nginx            soft    nofile          65535
      nginx            hard    nofile          65535container_commands:
  01-mv:
    command: "mv -f /opt/elasticbeanstalk/#etc#nginx#optimized-nginx.conf /tmp/deployment/config/#etc#nginx#nginx.conf"

enablewebsockets.config

container_commands:
  enable_websockets:
    command: |
      sed -i '/\s*proxy_set_header\s*Connection/c \
              proxy_set_header Upgrade $http_upgrade;\
              proxy_set_header Connection "upgrade";\
              ' /tmp/deployment/config/#etc#nginx#conf.d#00_elastic_beanstalk_proxy.conf

proxy.config

files:
  /etc/nginx/conf.d/proxy.conf:
    mode: "000644"
    owner: root
    group: root
    content: |
      # Elastic Beanstalk Custom Proxy to Redirect HTTP -> HTTPS# Elastic Beanstalk managed configuration file
      # Some configuration of nginx can be by placing files in /etc/nginx/conf.d
      # using Configuration Files.
      # http://docs.amazonwebservices.com/elasticbeanstalk/latest/dg/customize-containers.html
      #
      # Modifications of nginx.conf can be performed using container_commands to modify the staged version
      # located in /tmp/deployment/config/etc#nginx#nginx.confupstream nodejs {
          server 127.0.0.1:8081;
          keepalive 256;
      }server {
          listen 8080;if ($http_x_forwarded_proto = "http") {
            return 301 https://$host$request_uri;
          }if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2})") {
              set $year $1;
              set $month $2;
              set $day $3;
              set $hour $4;
          }
          access_log /var/log/nginx/healthd/application.log.$year-$month-$day-$hour healthd;
          access_log  /var/log/nginx/access.log  main;location / {
              proxy_pass  http://nodejs;
              proxy_set_header   Connection "";
              proxy_http_version 1.1;
              proxy_set_header        Host            $host;
              proxy_set_header        X-Real-IP       $remote_addr;
              proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
              proxy_set_header Upgrade $http_upgrade;
              proxy_set_header Connection "upgrade";
          }gzip on;
      gzip_comp_level 4;
      gzip_types text/html text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;}container_commands:
 removeconfig:
    command: "rm -f /tmp/deployment/config/#etc#nginx#conf.d#00_elastic_beanstalk_proxy.conf /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf"

Misc

Here’s everything else

ebtimeout.config

option_settings:
  - namespace: aws:elasticbeanstalk:command
    option_name: Timeout
    value: 1200

The default is 90, I just wanted to make sure the uploads go through and the setup process completes before it times out.

customnpminstall.config

.files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/55npm_install.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash
      # Custom npm install to work with Meteor/s build command
      export USER=root
      export HOME=/tmp
      export NODE_PATH=`ls -td /opt/elasticbeanstalk/node-install/node-* | head -1`/bin
      echo "------------------------------ — Installing NPM modules for Meteor  — -----------------------------------"
      OUT=$([ -d "/tmp/deployment/application" ] && cd /tmp/deployment/application/programs/server && $NODE_PATH/npm install --production) || error_exit "Failed to run npm install.  $OUT" $?
      echo $OUT

So the file structure I’d have is

directory
  _aws
    .ebextensions
      All the .config files from aboice  app
    Meteor stuff  build
    bundle
      The Node app

Now I would $ cd build/bundle

Run eb init then eb create app-env-alb and when it asks you to chose which application to chose from, go with 2 — application.

That will create a .elasticbeanstalk folder with config.yml in there. What I did next was take that out and put it inside the _aws folder I created earlier.

The config.yml file should look something like this

branch-defaults:
  default:
    environment: app-env-alb
environment-defaults:
  app-env-alb:
    branch: null
    repository: null
global:
  application_name: app
  default_ec2_keyname: if you have one
  default_platform: 64bit Amazon Linux 2016.09 v3.3.0 running Node.js
  default_region: us-west-2
  profile: eb-cli
  sc: null

Finally, create a package.json file inside _aws , this will be the main config for the app when it uploads to EB

{
  "name": "app",
  "version": "1.0.0",
  "scripts": {
    "start": "node main.js"
  },
  "dependencies": {
    "bcrypt": "*",
    "fibers": "*",
    "forever": "*",
    "semver": "*",
    "source-map-support": "*",
    "underscore": "*"
  }
}

Now you should have

directory
  _aws
    .ebextensions
      All the .config files from aboice
    .elasticbeanstalk
      config.yml
    package.json  app
    Meteor stuff  build
    bundle
      The Node app

Finally, I created a deploy script and put it inside directory/deploy-aws.sh

#!/bin/bash
 
#===============================================================================
# DESCRIPTION:
#===============================================================================
# This script creates a build of the project ready to be uploaded to beanstalk.
# Yoy must put this file in the directory above your project.
# Requires pyton 2.7.x
 
#===============================================================================
# COMMON ISSUES:
#===============================================================================
# -If you upload the output to a sample application, it will fail.
# -Version format must be 0.0.0
 
#===============================================================================
# CONSTANTS
#===============================================================================
CURRENT_VERSION="1.0.0"
OUTPUT_NAME="app"
PRODUCTION_SETTINGS_JSON="./app/settings-aws.json"
PROJECT_DIRECTORY="./app"
OUTPUT_DIRECTORY="./build"
ROOT_URL="https://app.com"
MONGO_URL="Your Mongo URL"
MONGO_OPLOG_URL="Your OPLOG URL"
AWS_SECRET_KEY="Secret Key"
AWS_ACCESS_KEY_ID="Key ID"
PORT="8081"#===============================================================================
# SAY HELLO
#===============================================================================
initial_directory=$(pwd)  #  This file's local path
clear
echo "COOKING OUTPUT"
echo "========================================================="
 
#===============================================================================
# RAW PROJECT BUILD
#===============================================================================
echo "> BUILDING RAW PROJECT"
cd $initial_directory
cd $PROJECT_DIRECTORY
rm -f  "../build/$OUTPUT_NAME-$CURRENT_VERSION.zip"
rm -f -R "../build/bundle"
npm install --production
meteor build --directory ../build/ --architecture os.linux.x86_64
 
#===============================================================================
# SET PRODUCTION ENVIRONMENT VARIABLES
#===============================================================================
cd $initial_directory
json=`cat $PRODUCTION_SETTINGS_JSON`
cd $OUTPUT_DIRECTORY/bundle
mkdir -p .ebextensions
cp -r ../../_aws/.ebextensions/customnpminstall.config ./.ebextensions # copy npm install options
cp -r ../../_aws/.ebextensions/enablewebsockets.config ./.ebextensions # copy websocket options
cp -r ../../_aws/.ebextensions/nginxoptimization.config ./.ebextensions # copy nginx options
cp -r ../../_aws/.ebextensions/application-load-balancer.config ./.ebextensions # copy alb options
cp -r ../../_aws/.ebextensions/alb-access-logs.config ./.ebextensions # copy alb options
cp -r ../../_aws/.ebextensions/alb-default-process.config ./.ebextensions # copy alb options
cp -r ../../_aws/.ebextensions/alb-secure-listener.config ./.ebextensions # copy alb options
cp -r ../../_aws/.ebextensions/ebtimeout.config ./.ebextensions # copy alb options
cp -r ../../_aws/.ebextensions/proxy.config ./.ebextensions # copy alb options
echo "option_settings:" >> .ebextensions/environment.config
echo "  - namespace:  aws:elasticbeanstalk:application:environment" >> .ebextensions/environment.config
echo "    option_name: MONGO_URL" >> .ebextensions/environment.config
echo "    value: $MONGO_URL" >> .ebextensions/environment.config
echo "  - namespace:  aws:elasticbeanstalk:application:environment" >> .ebextensions/environment.config
echo "    option_name: MONGO_OPLOG_URL" >> .ebextensions/environment.config
echo "    value: $MONGO_OPLOG_URL" >> .ebextensions/environment.config
echo "  - namespace:  aws:elasticbeanstalk:application:environment" >> .ebextensions/environment.config
echo "    option_name: AWS_SECRET_KEY" >> .ebextensions/environment.config
echo "    value: $AWS_SECRET_KEY" >> .ebextensions/environment.config
echo "  - namespace:  aws:elasticbeanstalk:application:environment" >> .ebextensions/environment.config
echo "    option_name: AWS_ACCESS_KEY_ID" >> .ebextensions/environment.config
echo "    value: $AWS_ACCESS_KEY_ID" >> .ebextensions/environment.config
echo "  - namespace:  aws:elasticbeanstalk:application:environment" >> .ebextensions/environment.config
echo "    option_name: PORT" >> .ebextensions/environment.config
echo "    value: $PORT" >> .ebextensions/environment.config
echo "  - namespace:  aws:elasticbeanstalk:application:environment" >> .ebextensions/environment.config
echo "    option_name: ROOT_URL" >> .ebextensions/environment.config
echo "    value: $ROOT_URL" >> .ebextensions/environment.config
echo "  - namespace:  aws:elasticbeanstalk:application:environment" >> .ebextensions/environment.config
echo "    option_name: METEOR_SETTINGS_FILE" >> .ebextensions/environment.config
echo "    value: /tmp/settings.json" >> .ebextensions/environment.config
echo "  - namespace:  aws:elasticbeanstalk:container:nodejs" >> .ebextensions/environment.config
echo "    option_name: ProxyServer" >> .ebextensions/environment.config
echo "    value: nginx" >> .ebextensions/environment.config
echo "    option_name: GzipCompression" >> .ebextensions/environment.config
echo "    value: true" >> .ebextensions/environment.config
echo "  - namespace:  aws:elasticbeanstalk:container:nodejs:staticfiles" >> .ebextensions/environment.config
echo "    option_name: /public" >> .ebextensions/environment.config
echo "    value: /public" >> .ebextensions/environment.config
echo "files:" >> .ebextensions/environment.config
echo " '/tmp/settings.json':" >> .ebextensions/environment.config
echo "    content : |" >> .ebextensions/environment.config
echo "        "$json >> .ebextensions/environment.config
chmod 444 .ebextensions/environment.config
echo "> ADDING 'settings.json' AS ENV VAR"
 
#===============================================================================
# CREATE PACKAGE.JSON
#===============================================================================cp -r ../../_aws/package.json ./ # copy package.json
mkdir -p .elasticbeanstalk
cp -r ../../_aws/.elasticbeanstalk/config.yml ./.elasticbeanstalk # copy elastic beanstalk configcd $initial_directory
cd $OUTPUT_DIRECTORY/bundlerm -f -r ./programs/server/npm/node_modules/bcryptecho "> ADDING 'package.json'"
 
#===============================================================================
# ZIP OUTPUT
#===============================================================================
cd $initial_directory
cd $OUTPUT_DIRECTORY/bundleeb deploy

The METEOR_SETTINGS_FILE gets created and uploaded to /tmp/settings.json so that you can access it within Meteor to set up your server settings. If you know of a better workaround for both client and server settings, let me know!

That, hopefully will deploy everything to Elastic Beanstalk. Let me know if you run into any errors, this process was executed in chunks, so I am not sure if it will work 100% from the start, but it’s my best shot!

AWS Console

I realized, one thing I forgot to do was, make sure that the Node version is 4.6.1 , otherwise, if you have 6.*.* your app will be at 100% CPU in idle state.

This is what my container options look like

Here’s a screenshot from my Application Load Balancer to make sure everything is set up

Conclusion

This was a frustrating process, getting everything together, figuring out all the different configurations, so if I can help someone achieve this with less headaches and less coffee, job well done then!

Huge thanks to AWS Support, those guys are awesome, and all the articles, StackOverflow questions/responses I went through countless times.

Have an awesome day.

416serg’s blog

Step-by-step guide to deploy a Meteor Application to AWS Elastic Beanstalk in Production

Meteor Setup

AWS Stuff

Application Load Balancer

Nginx

Misc

AWS Console

Conclusion

Sign up to discover human stories that deepen your understanding of the world.

Free

Membership

Published in 416serg’s blog

Written by Sergiy Dybskiy

Responses (3)