My first attempt to serverless API hosting

Here, I’ll demonstrate how to host an HTTP api call on AWS using Lambda and Api Gateway.

Servers are good. But there are a lot of boring tasks when it comes to maintaining a server. If I just want to publish a simple API online, serverless seems a better choice. Let’s try create an HTTP api using AWS Lambda and ApiGateway.

Create a lambda function

Create a simple python function which prints the client’s source IP. Don’t worry when testing the function in lambda console fails.

import json

def lambda_handler(event, context):
    return {
        'statusCode': 200,
        'headers': {'Content-type': 'application/json'},
        'body': json.dumps({'your-ip': event['requestContext']['http']['sourceIp']})
    }

Create an API gateway

On the api gatetway consone, create a simple HTTP API. Give the API a name and then leave everything else default. Once created, copy the URL of the apigateway from console. That is where the API is exposed.

Click create to create the API

Next, create a route for the apigateway. Here I create a route for GET to go to /. This API call will be read only so only GET request is needed.

For authorization, I’ll let it remain open which is the default setting.

Next, create an integration. Choose “ANY /” route, and set the integration target to the above lambda function. Ensure “Grant API Gateway permission to invoke your Lambda function” is enabled.

Moment of truth

Open the apigateway URL and the message is displayed. The actual result has been redacted.

▶ http https://85nd6u7qdi.execute-api.ap-southeast-1.amazonaws.com/  
HTTP/1.1 200 OK
Apigw-Requestid: dhO71jfiSQ0EPPg=
Connection: keep-alive
Content-Length: 29
Content-Type: application/json
Date: Fri, 09 Apr 2021 14:03:30 GMT

{
    "your-ip": "1.2.3.4"
}

Adding a custom domain name

The last step is to put this API on a custom domain name. Initially, I just created a cname on Cloudflare but that didn’t work. I even configured a CORS to allow the custom domain. Turns out I need to use the custom domain setting in api gateway.

First, create a certificate on ACM. Next, create a new custom domain on api gateway. Under API mappings, map the custom domain to my API. Finally, create a CNAME record, pointing the custom domain to the API gateway domain name (which is different from the apigateway URL).

▶ http https://myip.one27.cf
 HTTP/1.1 200 OK
 Apigw-Requestid: dkAVOhMpSQ0EJPg=
 Connection: keep-alive
 Content-Length: 29
 Content-Type: application/json
 Date: Sat, 10 Apr 2021 10:14:31 GMT
 {
     "your-ip": "1.2.3.4"
 }

My adventure on MikroTik

Recently, I came across the MikroTik routers and so happen I need to troubleshoot a site-to-site VPN issue.

Recently, I came across the MikroTik routers and so happen I need to troubleshoot a site-to-site VPN issue. MikroTik providers the RouterOS for anyone to install in a virtual environment. There is also an AMI on AWS. Here I’ll demonstrate how to get it to connect to AWS VPN in no time.

Continue reading “My adventure on MikroTik”

Connecting to VPC with AWS Client VPN

Connect to your VPC with AWS Client VPN.

AWS Client VPN allows users to connect to their VPC securely over the Internet. On AWS side, we configure client VPN endpoint. On the user side, we install the AWS VPN client software.

Here is a diagram demonstrating how we can use AWS Client VPN to connect to multiple VPCs.

Read on to see how it’s set up.

Continue reading “Connecting to VPC with AWS Client VPN”

Add security group to all ec2 instances

If you ever need to take over management of an AWS farm, it’s very likely you will need to attach SG to all instances. Be that for monitoring or access. Here is a bash script to add 1 SG to all instances.

You will need to first setup a profile on awscli. Then run the script with the profile name as first argument, and the SG id as the second.

Note: The script will not work on instances with multiple NICs

#!/usr/bin/env bash
#
# script to add 1 SG to all instances.
# this scripts takes 2 arguments, first is the aws profile name, second is the SG to add.
# e.g. ./add-sg.sh acme sg-1234567
#
# you will need awscli for this script to work, and an aws profile
# associated with an IAM user with the AmazonEC2FullAccess policy

export AWSPROFILE=$1
export ADDSG=$2

doit() {
    echo "Checking $1..."
    SG=$(aws --profile=$AWSPROFILE ec2 describe-instances --instance-ids $1 --output json | jq ".[][].Instances[].SecurityGroups[].GroupId" -r | xargs)
    echo "Existing SGs: $SG"
    if [[ $SG == *$ADDSG* ]]; then
        echo "$ADDSG already associated, do nothing"
    else
      aws --profile=$AWSPROFILE ec2 modify-instance-attribute --dry-run --instance-id $1 --groups $SG $ADDSG
      echo "New SGs: $(aws --profile=$AWSPROFILE ec2 describe-instances --instance-ids $1 --output json | jq ".[][].Instances[].SecurityGroups[].GroupId" -r | xargs)"
    fi
}

export -f doit

aws --profile=$AWSPROFILE ec2 describe-instances --output json \
| jq ".[][].Instances[].InstanceId" -r | parallel -j10 doit

AWS Application Load Balancer Request Tracing

ALB is a layer7 load balancer on AWS. It offers more features than ELB. You can find out more about it on https://aws.amazon.com/elasticloadbalancing/applicationloadbalancer/

Recently, I was asked to look into a random 504 issue with ELB. Because there is not a lot to configure on ELB and it’s pretty much a black box, I wasn’t making any headway there. The client believes their code is solid and this is an ELB issue. In fact, others have reported random 504 problems and they believe when ELB reloads, states are lost.

In this post, I want to demonstrate the request tracing feature which will be handy for troubleshooting timeout errors. ALB inserts a header (X-Amzn-Trace-Id) into each request with a unique id. This stamp allows admins or developers to trace the request to web servers.

Continue reading “AWS Application Load Balancer Request Tracing”

Let CloudWatch monitor your disk and memory utilisation

It is possible to use CloudWatch to monitor your disk and memory usage. There are very good information on AWS – http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/mon-scripts-perl.html

For the impatient, here is how to get it done.
Continue reading “Let CloudWatch monitor your disk and memory utilisation”