Advanced Topics Ahead

One of the major benefits of using Skyliner is it allows you to run your applications on a cloud-native, best-of-breed AWS architecture without actually having to do that work yourself. While in-depth knowledge of AWS technologies will help you use Skyliner more efficiently, it’s certainly not required.


All of the resources that Skyliner creates to run your applications are created and managed via CloudFormation. CloudFormation’s main concept is a template, which is a declarative manifest of what resources should be created, their properties, and the way they fit together.

Templates are used to create or update stacks, which are sets of the actual AWS resources described in the template. Existing stacks can be updated by changing the stack’s template, at which point CloudFormation will calculate and perform the set of steps (e.g. creating new resources or deleting old ones) required to make the stack consistent with the new template.

Skyliner uses CloudFormation to provide a consistent, common, and centralized interface for making changes to your AWS account. CloudFormation records all changes made to its stacks and serves as a central ledger describing all of the changes made to your AWS account.

Installing Skyliner

When you install Skyliner into an AWS region, it creates three stacks: a base stack and two environment stacks.

Common Resources

Creating the base stack does the following:

  • Provisions a KMS customer key. This key is used to encrypt CloudTrail audit logs, Skyliner configuration parameters, and other sensitive information.
  • Creates an S3 bucket for logs. This bucket is configured to accept ELB logs, S3 logs, and CloudTrail logs.
  • Creates an S3 bucket for storage. All access to this bucket is logged to the log bucket. In addition to serving as storage for your applications, this bucket is used to store Docker images.
  • Creates a cross-region CloudTrail audit log. This audit log records all AWS API access (including access by Skyliner on your behalf) and stores the entries in your S3 log bucket, encrypted with your KMS key and in a tamper-proof format.
  • CloudTrail entries are also sent to a dedicated CloudWatch Logs log group, allowing you to create alerts for e.g. AWS API authentication failures.

Environment Resources

Skyliner creates two environment stacks–one for QA and one for Production. Each stack contains the following resources:

  • A VPC with a unique, non-overlapping CIDR block, allowing you to peer with non-Skyliner VPCs.
  • For each VPC-enabled availability zone:
    • a VPC subnet for ELB instances
    • a VPC subnet for application instances
    • a non-routable VPC subnet for RDS/ElastiCache/Redshift instances
  • Subnet groups for RDS, ElastiCache, and Redshift, allowing you to easily place databases in the correct subnets.

Subnet sizes are calculated based on the number of availability zones in the region. For a three-AZ region like us-west-2, the application subnet would be a /19, and the ELB and database subnets a /20 each.

Working With Applications

Creating An Application

When you create an application in Skyliner, Skyliner creates two copies of an application stack–one for QA and one for production. Each application stack contains the following:

  • An Application Load Balancer, which supports both HTTP/2 and WebSockets, configured to send logs to your S3 log bucket. Connection draining is enabled, which allows for zero-downtime deploys. If your application is configured to accept HTTPS connections, the ELB is configured to use Amazon’s most recent TLS security policy (i.e. ELBSecurityPolicy-2015-05).
  • An ELB target group, which runs health checks on your application.
  • An EC2 security group for the ELB, allowing only HTTP/HTTPS/HTTP+HTTPS connections, depending on your application’s configuration.
  • An EC2 security group for application instances, allowing only SSH connections and HTTP connections on the configured local port from the ELB.
  • An EC2 security group for e.g. RDS instances, allowing only connections from application instances.
  • An IAM policy containing, in addition to any custom IAM statements, the following permissions:
    • read-only access to the application’s Docker images on the S3 storage bucket
    • full access to the /{environment}/{application} portion of the S3 storage bucket
    • encrypt and decrypt access to the KMS key for decrypting configuration parameters
    • read and write access to the application’s CloudWatch Logs log group
  • An IAM role and instance profile allowing all application instances to run with the role’s permissions.

Deploying An Application

Each time you deploy an application with Skyliner, it create a release stack containing the following:

  • An autoscaling group configured to maintain the given number of application instances.
  • A launch configuration specifying the application’s IAM instance profile, the latest Amazon Linux AMI, the configured EC2 instance type, the application’s security group, the SSH key pair, and a userdata script which performs the following tasks on instance launch:
    • applies all outstanding package upgrades
    • installs and configures Docker
    • installs configures the AWS Logs and SSM agents
    • decrypts the configuration parameters and stores their plaintext values on disk
    • downloads the Docker image from S3 and loads it into Docker
    • runs the Docker image using host networking and a restart policy

Why Not ECS/Kubernetes/Swarm/etc.?

Our goal with Skyliner is to build a product that is both reliable and accessible. The container orchestration ecosystem is incredibly promising but also incredibly young. In contrast, EC2 has seen more than a decade of investment in reliability and operability. It has a clean, well-defined control plane and peerless support for ad-hoc debugging when things go wrong with your application.

Amazon’s sole container offering–Elastic Container Service–is only semi-managed. In order to use ECS you need to run and manage a cluster of EC2 instances running the ECS agent. As a result, adopters of ECS are responsible not only for the operation of their applications but also for the operation of their ECS fleet. To us, that was an unacceptable tradeoff for a product whose promise is bringing simplicity and order to the AWS ecosystem.

That said, our model intentionally maps well to a container orchestration approach. If Amazon releases something akin to Google’s Container Image–a fully-managed container solution–expect us to move quickly in adopting that.

Questions? Feel free to email support or join us on Slack.