Deploying a .NET Core API to AWS Lambda & API Gateway using Bitbucket

After a transition to Bitbucket for our repository hosting last year, we’ve been setting up more and more CI/CD using Bitbucket Pipelines. I recently converted a .NET Core API project that was deploying to an auto-scaling group of EC2s into a project that could be deployed in AWS Lambda behind API Gateway using Bitbucket. The web project that consumes this API is written in Angular.

As a shop that leverages automated deployments in multiple environments, I found documentation on the web to be in short supply other than very basic “deploy using VS Studio” articles.

As part of updating the API project to Lambda, I did make my LambdaEntryPoint reference APIGatewayHttpApiV2ProxyFunction and the serverless template event of type HttpApi. A guide to these updates can found here: One Month Update to .NET Core 3.1 Lambda

In this post, I provide some snippets of YAML code which I found out in the world that didn’t work, and also what ultimately did.

Jump to What Worked

What Didn’t Work

aws-sam-deploy

caches:
  - dotnetcore
steps:
  - export PROJECT_NAME=this-dotnet-project
  - dotnet restore
  - dotnet build $PROJECT_NAME
  - pipe: atlassian/aws-sam-deploy:1.5.0

When trying to use the aws-sam-deploy pipe, I wasn’t able to leverage enough options or get the API to run the .NET code successfully. The API Gateway endpoint was running and hitting Lambda, but I was getting system errors I just couldn’t resolve.

Using project appsettings files

Since appsettings.json files contains secrets, we don’t check them into the repo. At some point I was receiving these errors, and I realized that the appsettings files weren’t getting deployed correctly.

run_dotnet(dotnet_path, &args) failed

Could not find the required 'this-dotnet-project.deps.json'. This file should be present at the root of the deployment package.: LambdaException

We ended up injecting the appsettings content using the AWS Parameter Store with aws ssm get-parameter.

dotnet publish, zip, and aws-lambda-deploy

- apt-get update && apt-get install --yes zip
- dotnet restore
- dotnet publish ${API_PROJECT_NAME} --output "./publish"  --framework "netcoreapp3.1" /p:GenerateRuntimeConfigurationFiles=true --runtime linux-x64 --self-contained false
- curl -o /bin/jp -L https://github.com/jmespath/jp/releases/download/0.1.3/jp-linux-amd64 && chmod a+x /bin/jp
- aws ssm get-parameter --name "/this-project/api/dev/appsettings" --with-decryption --region us-east-1 | /bin/jp -u "Parameter.Value" | base64 -d > ./publish/appsettings.json
- zip -r -j package.zip publish/*         
- pipe: atlassian/aws-lambda-deploy:1.5.0
  variables:
     AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
     AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
     AWS_DEFAULT_REGION: ${AWS_REGION}
     FUNCTION_NAME: "this-dotnet-project-AspNetCoreFunction-LZj5pbvV0GRT"
     COMMAND: "update"
     ZIP_FILE: "$BITBUCKET_CLONE_DIR/package.zip"

We have some single Lambda functions (not behind API Gateway) that use this method for deploying. It works great. I tried using this method pushing to a function that was built with a stack published via VS Studio. No luck. It’s possible there was a problem with the stack that was built, but I think this package wasn’t exactly right.

What Works

The following is the pipeline for a single branch, our develop branch. I haven’t yet refactored using template steps, but this is easier to read through for this article anyway.

I have scrubbed the contents of this YAML, but the repo contains:

• Root (.sln file)
  • .Net Core API project directory (named this-dotnet-project here)
  • Angular web project directory (named this-web-project here)

pipelines:  
  branches:
    develop:
      - step:
          name: API (.Net Core) Build & Deploy 
          image: mcr.microsoft.com/dotnet/core/sdk:3.1
          deployment: Develop
          script:
          - apt-get update && apt-get install -y zip && apt-get install -y awscli
          - dotnet tool install -g Amazon.Lambda.Tools
          - export PATH="$PATH:/root/.dotnet/tools"
          - curl -o /bin/jp -L https://github.com/jmespath/jp/releases/download/0.1.3/jp-linux-amd64 && chmod a+x /bin/jp
          - aws ssm get-parameter --name "/this-project/api/dev/appsettings" --with-decryption --region us-east-1 | /bin/jp -u "Parameter.Value" | base64 -d > ./this-dotnet-project/appsettings.json
          - cd this-dotnet-project/
          - dotnet lambda deploy-serverless --aws-access-key-id ${AWS_ACCESS_KEY_ID} --aws-secret-key ${AWS_SECRET_ACCESS_KEY} --region ${AWS_REGION} --configuration "Development" --framework "netcoreapp3.1" --runtime linux-x64 --s3-bucket $API_S3_BUCKET --stack-name $API_STACK_NAME --stack-wait true
      - step:
          name: Web (Angular) Build
          image: atlassian/default-image:2
          caches:
          - node
          script:
	      - cd this-web-project #The angular project is currently in a subfolder in the same repo
          - nvm install 12
          - nvm use 12
          - npm install @angular/cli
          - npm run build:dev
          artifacts: # defining the artifacts to be passed to each future step.
          - dist/**
      - step:
          name: Web (Angular) Deploy
          script:
          - pipe: atlassian/aws-s3-deploy:0.5.0
            variables:
              AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID} 
              AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
              AWS_DEFAULT_REGION: ${AWS_REGION}
              S3_BUCKET: ${WEB_S3_BUCKET_DEV}
              LOCAL_PATH: "this-web-project/dist/this-project-output-path/"

apt-get install and dotnet tool install

Items that were quickly apparent as missing before adding them, more of a “duh” leaving them out when trying so many things.

dotnet lambda deploy-serverless

This was the big command that mostly got things working. I finally found this is effectively what’s happening when you deploy the API project from VS Studio.

–stack-wait true

In Bitbucket, without this, the build shows as successful when the stack build is kicked off. By adding this flag, bitbucket will wait for the full build or update before continuing.