Building a Highly Available, Scalable Web

Application

Overview and objectives

Throughout various AWS Academy courses, you have completed hands-on labs. You have
used different AWS services and features to create compute instances, install operating
systems (OSs) and software, deploy code, and secure resources. You practiced how to enable
load balancing and automatic scaling, and how to architect for high availability to build
simple, lab-specific applications.

In this project, you’re challenged to use familiar AWS services to build a solution without
step-by-step guidance. Specific sections of the assignment are meant to challenge you on
skills that you have acquired throughout the learning process.

By the end of this project, you should be able to do the following:

 Create an architectural diagram to depict various AWS services and their interactions
with each other.
 Estimate the cost of using services by using the AWS Pricing Calculator.
 Deploy a functional web application that runs on a single virtual machine and is
backed by a relational database.
 Architect a web application to separate layers of the application, such as the web
server and database.
 Create a virtual network that is configured appropriately to host a web application that
is publicly accessible and secure.
 Deploy a web application with the load distributed across multiple web servers.
 Configure the appropriate network security settings for the web servers and database.
 Implement high availability and scalability in the deployed solution.
 Configure access permissions between AWS services.

The lab environment and monitoring your budget

This environment is long lived. When the session timer runs to 0:00, the session will end,
but any data and resources that you created in the AWS account will be retained. If you later
launch a new session (for example, the next day), you will find that your work is still in the
lab environment. Also, at any point before the session timer reaches 0:00, you can choose
Start Lab again to extend the lab session time.

Important: Monitor your lab budget in the lab interface. When you have an active lab
session, the latest known remaining budget information displays at the top of this screen. This
data comes from AWS Budgets, which typically updates every 8–12 hours. Therefore, the
remaining budget that you see might not reflect your most recent account activity. If you
exceed your lab budget, your lab account will be disabled, and all progress and
resources will be lost. Therefore, it’s important for you to manage your spending.
AWS service restrictions
In this lab environment, access to AWS services and service actions might be restricted to the
ones that are needed to complete the lab instructions. You might encounter errors if you
attempt to access other services or perform actions beyond the ones that are described in this
lab.

Scenario
Example University is preparing for the new school year. The admissions department has
received complaints that their web application for student records is slow or not available
during the peak admissions period because of the high number of inquiries.

You are a cloud engineer. Your manager has asked you to create a proof of concept (POC) to
host the web application in the AWS Cloud. Your manager would like you to design and
implement a new hosting architecture that will improve the experience for users of the web
application. You’re responsible for building the infrastructure to host the student records web
application in the cloud.

Your challenge is to plan, design, build, and deploy the web application to the AWS Cloud in
a way that is consistent with best practices of the AWS Well-Architected Framework. During
the peak admissions period, the application must support thousands of users, and be highly
available, scalable, load balanced, secure, and high performing.

The following image shows an example of the student records web application. The site lists
records of students who have applied for admission to the university. Users can view, add,
delete, and modify student records.

Solution requirements
The solution must meet the following requirements:

 Functional: The solution meets the functional requirements, such as the ability to
view, add, delete, or modify the student records, without any perceivable delay.
 Load balanced: The solution can properly balance user traffic to avoid overloaded or
underutilized resources.
 Scalable: The solution is designed to scale to meet the demands that are placed on the
application.
 Highly available: The solution is designed to have limited downtime when a web
server becomes unavailable.
 Secure:
o The database is secured and can’t be accessed directly from public networks.
o The web servers and database can be accessed only over the appropriate ports.
o The web application is accessible over the internet.
 o The database credentials aren’t hardcoded into the web application.
 Cost optimized: The solution is designed to keep costs low.
 High performing: The routine operations (viewing, adding, deleting, or modifying
records) are performed without a perceivable delay under normal, variable, and peak
loads.

Assumptions

This project will be built in a controlled lab environment that has restrictions on services,
features, and budget. Consider the following assumptions for the project:

 The application is deployed in one AWS Region (the solution does not need to be
multi-Regional).
 The website does not need to be available over HTTPS or a custom domain.
 The solution is deployed on Ubuntu machines by using the JavaScript code that is
provided.
 Use the JavaScript code as written unless the instructions specifically direct you to
change the code.
 The solution uses services and features within the restrictions of the lab environment.
 The database is hosted only in a single Availability Zone.
 The website is publicly accessible without authentication.
 Estimation of cost is approximate.

Disclaimer: A security best practice is to allow access to the website through the university
network and authentication. However, because you are building this application as a POC,
those features are beyond the scope of this project. You are encouraged to implement this
additional functionality.

Approach
Recommendation: Develop your project solution in phases. This will help you ensure that
basic functionality is working before the architecture becomes more complex. After the
application is working, you are encouraged to enhance the solution with additional
requirements.

Phase 1: Planning the design and estimating cost

In this phase, you will plan the design of your architecture. First, you will create an
architecture diagram.

Next, you will estimate the cost of the proposed solution, and present the estimate to your
educator. An important first step for any solution is to plan the design and estimate the cost.
As necessary, review the various components in the architecture to adjust the estimated cost.
Cost is an important factor when building a solution because cost can help to determine the
components and architecture pattern to use.

Note: You don't need to use the lab environment for this phase of the project, but you might
want to use it to refer to AWS services and features as you plan your design.

Task 1: Creating an architectural diagram

Create an architectural diagram to illustrate what you plan to build. Consider how you will
accomplish each requirement in the solution.

References

 AWS Architecture Icons: This site provides tools to draw AWS architecture diagrams.
 AWS Reference Architecture Diagrams: This site provides reference architecture
diagrams for a variety of use cases.

Task 2: Developing a cost estimate

Develop a cost estimate that shows the cost to run the solution in the us-east-1 Region for 12
months. Use the AWS Pricing Calculator for this estimate.

If required by your instructor, add your architectural diagram and cost estimate to
presentation slides. Your educator might want to evaluate this information as part of assessing
your work on this project. A presentation template is provided.

References

 What Is AWS Pricing Calculator?

 PowerPoint presentation template
Phase 2: Creating a basic functional web application

In this phase, you will start to build the solution. The objective of this phase is to have a
functional web application that works on a single virtual machine in a virtual network that
you create. By the end of this phase, you will have a POC to demonstrate hosting the
application on the AWS Cloud. You can then build upon your work in later phases.

Task 1: Creating a virtual network

Create a virtual network to host the web application.

Tip: Create networking resources such as a virtual private cloud (VPC) and subnets.

Reference

 AWS Academy Cloud Architecting – Lab: Creating a Virtual Private Cloud

Task 2: Creating a virtual machine

Create a virtual machine in the cloud to host the web application.

To install the required web application and database on the virtual machine, use the
JavaScript code from the following link: SolutionCodePOC

Tips:

 Use a compute service such as Amazon Elastic Compute Cloud (Amazon EC2).
 Use the latest Ubuntu Amazon Machine Image (AMI).

El código a poner en la plantilla es el siguiente:

#!/bin/bash -xe
apt update -y
apt install nodejs unzip wget npm mysql-server -y
#wget https://aws-tc-largeobjects.s3.us-west-2.amazonaws.com/CUR-TF-200-
ACCAP1-1-DEV/code.zip -P /home/ubuntu
wget https://aws-tc-largeobjects.s3.us-west-2.amazonaws.com/CUR-TF-200-
ACCAP1-1-79581/1-lab-capstone-project-1/code.zip -P /home/ubuntu
cd /home/ubuntu
unzip code.zip -x "resources/codebase_partner/node_modules/*"
cd resources/codebase_partner
npm install aws aws-sdk
mysql -u root -e "CREATE USER 'nodeapp' IDENTIFIED WITH
mysql_native_password BY 'student12'";
mysql -u root -e "GRANT all privileges on *.* to 'nodeapp'@'%';"
mysql -u root -e "CREATE DATABASE STUDENTS;"
mysql -u root -e "USE STUDENTS; CREATE TABLE students(
 id INT NOT NULL AUTO_INCREMENT,
name VARCHAR(255) NOT NULL,
address VARCHAR(255) NOT NULL,
city VARCHAR(255) NOT NULL,
state VARCHAR(255) NOT NULL,
email VARCHAR(255) NOT NULL,
phone VARCHAR(100) NOT NULL,
PRIMARY KEY ( id ));"
sed -i 's/.*bind-address.*/bind-address = 0.0.0.0/' /etc/mysql/mysql.conf.d/mysqld.cnf
systemctl enable mysql
service mysql restart
export APP_DB_HOST=$(curl http://169.254.169.254/latest/meta-data/local-ipv4)
export APP_DB_USER=nodeapp
export APP_DB_PASSWORD=student12
export APP_DB_NAME=STUDENTS
export APP_PORT=80
npm start &
echo '#!/bin/bash -xe
cd /home/ubuntu/resources/codebase_partner
export APP_PORT=80
npm start' > /etc/rc.local
chmod +x /etc/rc.local

Se va a ejecutar como vemos un mysql-server local a la máquina EC2, en lugar de una base
de datos RDS.

Generamos una nueva plantilla para lanzamiento de instancias:

Lanzamos instancia con grupo de seguridad personalizado para esta parte del reto (puertos
80 y 22 abiertos) y red vpc por defecto.
Task 3: Testing the deployment

Test the deployment of the web application to ensure it is accessible from the internet and
functional. Perform a few tasks, such as viewing, adding, deleting, or modifying records.

Tip: To access the web application, use the IPv4 address of the virtual machine.
Phase 3: Decoupling the application components

In this phase, you will continue building. The objective is to separate the database and the
web server infrastructure so that they run independently. The web application should run on a
separate virtual machine, and the database should run on the managed service infrastructure.

Task 1: Changing the VPC configuration

Update or re-create the virtual network components that are necessary to support hosting the
database separately from the application.

Note: You need private subnets in a minimum of two Availability Zones.

Reference

 AWS Academy Cloud Architecting – Lab: Creating a Virtual Private Cloud

Utilizaremos la VPC creada para el reto:

Task 2: Creating and configuring the Amazon RDS database

Create an Amazon Relational Database Service (Amazon RDS) database that runs a MySQL
engine. You can choose to create a provisioned instance or run it serverlessly.

Notes:

 Allow only the web application to access the database.

 Don't enable enhanced monitoring.
Reference

 AWS Academy Cloud Foundations – Lab: Build Your DB Server and Interact With
Your DB Using an App

Creamos un servidor de base de datos nuevo con un grupo de seguridad nuevo para dar
acceso a este servidor de BD solo a la aplicación web que vamos a desplegar.

Task 3: Configuring the development environment

Provision an AWS Cloud9 environment to run AWS Command Line Interface (AWS CLI)
commands in later tasks.

Notes:

 Use a t3.micro instance for the AWS Cloud9 environment.

 Use Secure Shell (SSH) to connect to the environment.

Reference

 Creating Cloud9 Environment

Creamos un entorno con Cloud9 como solicita el reto:

Task 4: Provisioning Secrets Manager

Use AWS Secrets Manager to create a secret to store the database credentials, and configure
the web application to use Secrets Manager.

Use Script-1 from the following link to create a secret in Secrets Manager by using the AWS
CLI: AWS Cloud9 Scripts

Note: This .yml file also contains scripts that you will use in later tasks.

Reference

 create-secret in the AWS CLI Command Reference for AWS Secrets Manager

Ejecutamos el siguiente script:

aws secretsmanager create-secret \

--name Mydbsecret \
--description "Database secret for web app" \
--secret-string "{\"user\":\"nodeapp\",\"password\":\"student12\",\"host\":\"bd-reto-
final.csoqwbi93wsu.us-east-1.rds.amazonaws.com\",\"db\":\"STUDENTS\"}"
NOTA: El secreto también se podría haber creado a mano en AWS Secrets Manager sin
necesidad de script como hemos visto en clase.

Task 5: Provisioning a new instance for the web server

Create a new virtual machine to host the web application.

To install the required web application on the virtual machine, use the JavaScript code from
the following link: Solution Code for the App Server

For the AWS Identity and Access Management (IAM) profile on the EC2 instance,
attach the existing LabInstanceProfile profile. This profile attaches an IAM role called
LabRole to the instance so that it can fetch the secret securely.

NOTA: este punto es crítico para poder acceder a los secretos.

Si no lo hemos hecho o no hemos asociado el rol LabRole a la plantilla, como era mi caso,
hay que detener la instancia y modificar el rol de IAM:
Note: Optionally, you can continue to use the existing virtual machine for the web
application. However, you will need to reconfigure the application to connect to Amazon
RDS.

Reference

 AWS Academy Cloud Foundations – Lab: Build Your DB Server and Interact With
Your DB Using an App
Creamos una nueva plantilla para incluir el script de instalación de los servidores web sin
necesidad de mysql-server, sino con un cliente de mysql que conectará con nuestra base de
datos RDS.

Lanzamos una instancia a partir de esta plantilla, con grupo de seguridad SGPhase2 (puertos
80 y 22 abiertos). Aún no se solicita nada relativo a escalabilidad.

Entramos en la máquina recién creada y comprobamos que el script se ha ejecutado

correctamente: node está instalado, la aplicación se ha descargado y la tenemos corriendo
via npm start:

No funciona en este momento porque es necesario conectarla con la nueva base de datos.
Task 6: Migrating the database

Migrate the data from the original database, which is on an EC2 instance, to the new Amazon
RDS database.

Use Script-3 from the AWS Cloud9 Scripts file (cloud9-scripts.yml) to migrate the original
data into the Amazon RDS database. Recall that you used a script from this file earlier to
create the secret in Secrets Manager.

Reference

 AWS Academy Cloud Architecting – Lab: Migrating a Database to Amazon RDS

Nos conectamos directamente a la máquina EC2 MaquinaRetoPhase2 y extraemos los datos

del sql con el siguiente comando:

mysqldump -u nodeapp -p --databases STUDENTS > data.sql

Hay que tener en cuenta que la base de datos externa en RDS no está aún conectada a la
red privada y solo accedida por nuestra aplicación web.

Creamos en nuestra base de datos externa el usuario cuyas credenciales se emparejan con
lo establecido en AWS Secrets Manager, por lo que ejecutamos línea a línea:

mysql -h bd-reto-final.csoqwbi93wsu.us-east-1.rds.amazonaws.com -u admin -p -e

"CREATE USER 'nodeapp' IDENTIFIED WITH mysql_native_password BY
'student12'";

mysql -h bd-reto-final.csoqwbi93wsu.us-east-1.rds.amazonaws.com -u admin -p -e

"CREATE DATABASE STUDENTS";

mysql -h bd-reto-final.csoqwbi93wsu.us-east-1.rds.amazonaws.com -u admin -p -e

"GRANT all privileges on STUDENTS.* to 'nodeapp'@'%';"
mysql -h bd-reto-final.csoqwbi93wsu.us-east-1.rds.amazonaws.com -u nodeapp -p
STUDENTS < data.sql

Task 7: Testing the application

Access the application and perform a few tasks to test it. For example, view, add, delete, and
modify student records.

Vamos a la máquina WebServerRetoFinal1 creada y abrimos el archivo

/home/ubuntu/resources/codebase_partner/app/config/config.js

Este archivo contiene la configuración de la aplicación que queremos correr y como

podemos ver, necesitamos introducir los parámetros de configuración tanto de la base de
datos como del secreto de AWS Secrets Manager.
Ponemos el host remoto a nuestro servidor de base de datos en RDS y el resto de
configuración. El nombre del secreto es correcto y podrá ser accedido si asociamos la
máquina EC2 con el perfil de IAM correcto LabInstanceProfile (LabRole).

// define default config, but allow overrides from ENV vars

let config = {
APP_DB_HOST: "bd-reto-final.csoqwbi93wsu.us-east-1.rds.amazonaws.com",
APP_DB_USER: "nodeapp",
APP_DB_PASSWORD: "student12",
APP_DB_NAME: "STUDENTS"
}

var AWS = require('aws-sdk');

var client = new AWS.SecretsManager({
region: "us-east-1"
});

const secretName = "Mydbsecret";

Ahora es turno de relanzar la aplicación. Sabemos que el script con el que iniciaba la
máquina arrancaba npm y lo dejaba corriendo en segundo plano. Necesitamos saber el
proceso que está corriendo para matarlo y lanzarlo de nuevo para así recuperar los nuevos
datos de acceso a base de datos externa que hemos configurado.

Iniciamos de nuevo la aplicación, en principio de forma manual en primer plano para ver
que todo funciona correctamente:

export APP_PORT=80 && npm start --ddd

Probamos en el navegador que con la IP pública accedemos correctamente y que los datos
de la base de datos están siendo accedidos:
Vemos que la aplicación funciona correctamente añadiendo un nuevo registro:

Y ya hacemos ctrl+c para cortar la aplicación en primer plano y la dejamos corriendo en

background:

export APP_PORT=80 && npm start &

Phase 4: Implementing high availability and scalability

In this phase, you will complete the design and fulfill the remaining solution requirements.
The objective is to use the key components that you created in earlier phases to build a
scalable and highly available architecture.

Task 1: Creating an Application Load Balancer

Launch a load balancer. The endpoint will be used to access your web application.

Tip: Use a minimum of two Availability Zones.

Reference

 AWS Academy Cloud Architecting – Lab: Creating a Highly Available Environment

Task 2: Implementing Amazon EC2 Auto Scaling

Create a new launch template, and use an Auto Scaling group to launch the EC2 instances
that host the web application.

To accomplish this, you can create an AMI from the running instance, or create a new AMI
and install the necessary packages and application code. Then, configure an Auto Scaling
group to use the load balancer.

Tips:

 Use a Target tracking policy.

 Set the Auto scaling group size according to your estimated requirements.
 You can use the default values (for example, for group size and CPU utilization)
initially and then adjust them later as needed.

Reference

 AWS Academy Cloud Architecting – Lab: Creating a Highly Available Environment

Creamos el grupo de autoescala como hemos visto en clase:
Una vez creado, vamos a la pestaña “Actividad” para ver cómo se están creando dos
instancias que forman parte del mínimo que hemos establecido:

Cuando pasan unos minutos, vemos que se han iniciado un par de instancias con la plantilla
especificada:
Como parte de este grupo de auto escala, se generó un balanceador de carga con la
siguiente configuración:

Para efectos de depuración, necesitamos una máquina de bastión para poder conectar a los
nodos privados y revisar si la aplicación se está ejecutando allí correctamente.

Creamos una nueva máquina EC2 metiendo este host dentro de la VPC del reto.
Una vez creado el host bastión, descargamos la parte privada de la clave vockey del
laboratorio:

Y la copiamos a este host que usaremos como Bastion:

Y ya podemos acceder a los nodos privados utilizando esta clave privada:

Gracias a la depuración podemos ver que nos equivocamos al elegir las subredes para el
grupo de auto escala, por lo que corregimos y seleccionamos las privadas para los nodos
que se auto generan:
Terminamos las dos instancias generadas para que se creen nuevas gracias al grupo de auto
escala pero con las subredes correctas.

Otro tema muy importante es configurar la plantilla de generación de instancias para que
se relacione la instancia con el perfil de IAM que permite acceder al Secrets Manager. De
esta forma, no habrá que terminar y arrancar manualmente las máquinas para configurar
esta relación EC2 / IAM. Podemos cambiar esta configuración en Detalles avanzados:
Task 3: Accessing the application

Access the application and perform a few tasks to test it. For example, view, add, delete, and
modify student records.

Cuando se despliegan las dos nuevas instancias, al estar correctamente asociadas al perfil
que permite acceder al Secrets Manager, se conecta de forma correcta a nuestra base de
datos externa en RDS.

Y probamos a añadir un nuevo estudiante para corroborar que todo funciona como debería:
Task 4: Load testing the application

Perform a load test on the application to monitor scaling.

Use Script-2 from the AWS Cloud9 Scripts file (cloud9-scripts.yml) to perform the load test.
Recall that you used scripts from this file in previous tasks.

Notes:

 Access the web application from the browser by using the load balancer URL.
 Use AWS Cloud9 to run the load testing scripts against the load balancer.

Reference

 loadtest Tool Repository on GitHub

Lanzamos la aplicación solicitada, pero no hay una política de escalado dinámica, así que
vemos como se carga mucho en requests el ALB.

Los argumentos del comando loadtest son:

 --rps 1000: Indica que se realizarán 1000 solicitudes por segundo (requests per
second) a la URL especificada.
 -c 500: Especifica que se utilizarán 500 conexiones simultáneas (concurrency) para
realizar las solicitudes.
 -k: Indica que se utilizará la opción de "keep-alive" para mantener abiertas las
conexiones TCP y reutilizarlas para enviar varias solicitudes.
 http://as-reto-final-1-1217415727.us-east-1.elb.amazonaws.com/students: Es la
URL de destino en la que se realizarán las solicitudes.
Pero para lograr que se creen nuevas instancias dependiendo de ciertas condiciones, habría
que crear una política de escalado dinámica en el grupo de autoescalado:
Terminamos el reto cerrando la BD para que en lugar de acceso externo solo se tenga
acceso por parte de la aplicación:

Dejamos acceso a la base de datos solo a los nodos que están en el SGWeb:
Y todo sigue funcionando correctamente con la infraestructura securizada al máximo: