Code Push to EC2
November 16, 2024 · AI Conversations
pipelines:
default:
– step:
name: Build & Deploy
caches:
– node
script:
– echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add –
– mkdir -p ~/.ssh
– chmod 700 ~/.ssh
– echo "$EC2_SSH_PUBLIC_KEY" > ~/.ssh/authorized_keys
– chmod 600 ~/.ssh/authorized_keys
– git clone https://your-bitbucket-repository-url
– cd your-project-directory
– npm install
– npm run build
– composer install –no-dev –optimize-autoloader
– rsync -avz –exclude=.git –exclude=node_modules ./ ubuntu@your-ec2-public-ip:/path/to/your/project
– ssh -o StrictHostKeyChecking=no ubuntu@your-ec2-public-ip "cd /path/to/your/project && php artisan migrate && php artisan config:cache"
CHange above so that nothing install just code push to ec2 whenever we push to bitbucket
image: node:21
pipelines:
default:
- step:
name: Deploy Code to EC2
caches:
- node
script:
- echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$EC2_SSH_PUBLIC_KEY" > ~/.ssh/authorized_keys
- chmod 600 ~/.ssh/authorized_keys
- git clone https://your-bitbucket-repository-url
- cd your-project-directory
- rsync -avz --exclude=.git --exclude=node_modules ./ ubuntu@your-ec2-public-ip:/path/to/your/project
- ssh -o StrictHostKeyChecking=no ubuntu@your-ec2-public-ip "cd /path/to/your/project && git pull origin main"
### Key Changes:
1. **Removed `npm install` and `composer install`**: As you don't want to install dependencies.
2. **`git pull` on EC2**: After pushing the code with `rsync`, I added a `git pull` to update the repository on EC2 with the latest changes.
This configuration ensures that only the latest code is pushed to EC2 whenever there's a push to Bitbucket.
– cd your-project-directory
image: node:21
pipelines:
default:
- step:
name: Deploy Code to EC2
caches:
- node
script:
- echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$EC2_SSH_PUBLIC_KEY" > ~/.ssh/authorized_keys
- chmod 600 ~/.ssh/authorized_keys
- rsync -avz --exclude=.git --exclude=node_modules ./ ubuntu@your-ec2-public-ip:/path/to/your/project
- ssh -o StrictHostKeyChecking=no ubuntu@your-ec2-public-ip "cd /path/to/your/project && git pull origin main"
### Key Adjustments:
1. **Removed `git clone`**: Since you want to push code directly without cloning the repository.
2. **Removed `cd your-project-directory`**: No need to change directories after cloning, since you're using `rsync` to directly transfer files into the correct location on the EC2 instance.
This will now simply push the changes to your EC2 instance and pull the latest code from the Bitbucket repository on the EC2 side.
### Here's how it works:
1. **SSH Key Pair**: You generate an SSH key pair, consisting of a private key and a public key.
– The **private key** is kept secure on the EC2 instance, typically in `~/.ssh/authorized_keys` for the user that you want to authenticate with (e.g., `ubuntu`).
– The **public key** is added to the Bitbucket repository's environment variables (or a separate secrets management service like Bitbucket Secrets) to allow the pipeline to authenticate with the EC2 instance.
2. **Environment Variable**: You set the `$EC2_SSH_PUBLIC_KEY` in your Bitbucket repository's environment variables, so when the pipeline runs, it can use this key to establish an SSH connection with your EC2 instance.
3. **Usage in Pipeline**: In the pipeline script, this public key is echoed to a file (`~/.ssh/authorized_keys`) on the EC2 instance, allowing the pipeline to authenticate and perform tasks (like `rsync` or `git pull`).
### How to Set It Up:
1. **Generate SSH Keys**: If you don't already have an SSH key pair, you can generate one using:
ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
This will create a private key (`id_rsa`) and a public key (`id_rsa.pub`).
2. **Add Public Key to EC2 Instance**:
– Copy the contents of the `id_rsa.pub` file to your EC2 instance's `~/.ssh/authorized_keys` file for the appropriate user (e.g., `ubuntu`).
3. **Add Public Key to Bitbucket Repository**:
– Copy the contents of the `id_rsa.pub` file.
– Go to your Bitbucket repository settings, and add it to the environment variables under `Pipelines` > `Repository variables`, setting the key to `EC2_SSH_PUBLIC_KEY`.
In your pipeline, this public key will be used to authenticate the connection. If you also want to use a private key on the EC2 instance for authentication, you can add it as an environment variable or use an AWS Secrets Manager or similar service for secure management of the key.
ec2 user- ubuntu
project folder on ec2- /var/www/share
host:ec2-16-170-254-93.eu-north-1.compute.amazonaws.com
image: node:21
pipelines:
default:
- step:
name: Deploy Code to EC2
caches:
- node
script:
- echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$EC2_SSH_PUBLIC_KEY" > ~/.ssh/authorized_keys
- chmod 600 ~/.ssh/authorized_keys
- rsync -avz --exclude=.git --exclude=node_modules ./ ubuntu@ec2-16-170-254-93.eu-north-1.compute.amazonaws.com:/var/www/share
- ssh -o StrictHostKeyChecking=no ubuntu@ec2-16-170-254-93.eu-north-1.compute.amazonaws.com "cd /var/www/share && git pull origin main"
### Key Points:
– **SSH Keys**: Ensure that your SSH private key (`$SSH_PRIVATE_KEY`) and public key (`$EC2_SSH_PUBLIC_KEY`) are set as environment variables in Bitbucket to allow SSH access to your EC2 instance.
– **rsync**: This will push the changes from your local repository to the `/var/www/share` directory on your EC2 instance, excluding `.git` and `node_modules`.
– **git pull**: After syncing the files, the pipeline runs `git pull` on your EC2 instance to fetch the latest code if any changes are made to the repository.
Make sure the SSH public key (`$EC2_SSH_PUBLIC_KEY`) on the Bitbucket environment matches the one in `~/.ssh/authorized_keys` on the EC2 instance.
Let me know if you need further adjustments!
image: node:21
pipelines:
default:
- step:
name: Deploy Code to EC2
caches:
- node
script:
- echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$EC2_SSH_PUBLIC_KEY" > ~/.ssh/authorized_keys
- chmod 600 ~/.ssh/authorized_keys
- rsync -avz --exclude=.git --exclude=node_modules ./ ubuntu@ec2-16-170-254-93.eu-north-1.compute.amazonaws.com:/var/www/share
- ssh -o StrictHostKeyChecking=no ubuntu@ec2-16-170-254-93.eu-north-1.compute.amazonaws.com "cd /var/www/share && git pull origin main"
This assumes that:
– The EC2 instance has already been configured to accept SSH connections from Bitbucket using the public key.
– The `SSH_PRIVATE_KEY` environment variable in Bitbucket will allow the pipeline to authenticate using your private key.
– The `git pull` command on EC2 ensures the latest code is synced if the repository has been updated.
Let me know if there's anything else you'd like to adjust!
### 1. **Ensure Node.js and npm are Up-to-Date**
Sometimes older versions of Node.js or npm can cause slower builds. Since you're using Node.js 21, ensure that it's properly installed and that the version of npm is also up-to-date.
You can check the versions on your EC2 instance by running:
node -v
npm -v
If the version is outdated, you can update npm:
sudo npm install -g npm@latest
### 2. **Cache Node Modules in Bitbucket Pipeline**
Caching your `node_modules` directory in your Bitbucket pipeline can save time on subsequent builds by reusing previously installed dependencies.
In your Bitbucket pipeline, add a cache step for `node_modules`:
pipelines:
default:
- step:
name: Deploy Code to EC2
caches:
- node
script:
- echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$EC2_SSH_PUBLIC_KEY" > ~/.ssh/authorized_keys
- chmod 600 ~/.ssh/authorized_keys
- rsync -avz --exclude=.git --exclude=node_modules ./ ubuntu@ec2-16-170-254-93.eu-north-1.compute.amazonaws.com:/var/www/share
- ssh -o StrictHostKeyChecking=no ubuntu@ec2-16-170-254-93.eu-north-1.compute.amazonaws.com "cd /var/www/share && npm install && npm run build"
This will use Bitbucket’s caching feature to store `node_modules` between builds, reducing the time spent reinstalling dependencies.
### 3. **Use a Build Tool Like Vite (if not already using)**
If you are not already using a modern build tool like Vite, it might be worth considering. Vite is much faster than Webpack and is optimized for modern JavaScript frameworks like Vue.js.
To replace `webpack` with Vite:
1. Install Vite and the Vue plugin:
npm install vite @vitejs/plugin-vue
2. Create a `vite.config.js` in your project:
import { defineConfig } from 'vite';
import vue from '@vitejs/plugin-vue';
export default defineConfig({
plugins: [vue()],
});
3. Update your `package.json` to use Vite:
"scripts": {
"dev": "vite",
"build": "vite build",
"serve": "vite preview"
}
4. Update your build pipeline to run `vite build` instead of `npm run build`.
### 4. **Optimize Your `node_modules` Directory**
There may be unused or unnecessary dependencies in your `node_modules`. Using a tool like `npm dedupe` can help reduce redundancy and potentially speed up your builds:
npm dedupe
### 5. **Use a More Powerful EC2 Instance**
If your EC2 instance is underpowered for your build requirements, it may struggle with the build process. You can consider upgrading to a larger EC2 instance type with more CPU and memory resources. Consider using instances like `t3.medium` or `t3.large` for better performance during builds.
### 6. **Build on a Separate CI Server (e.g., Bitbucket Pipelines)**
Instead of building on EC2, consider building the project on a dedicated CI server (like Bitbucket Pipelines or GitHub Actions) and only transferring the final build assets (e.g., `dist/`, `public/`, etc.) to EC2. This can offload the heavy build task from your EC2 instance and improve the deployment speed.
In the pipeline, after building the assets, you could use `rsync` to transfer only the necessary files to the EC2 server, skipping the `npm run build` step on EC2 altogether.
### 7. **Parallelize Build Tasks**
If the build process involves multiple steps that can be parallelized (e.g., transpiling CSS, JS, images, etc.), you could consider running those steps concurrently to reduce the overall build time. This can be achieved using tools like `npm-run-all` or through configurations in your build scripts.
—
### Example Optimized Pipeline:
pipelines:
default:
- step:
name: Deploy Code to EC2
caches:
- node
script:
- echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$EC2_SSH_PUBLIC_KEY" > ~/.ssh/authorized_keys
- chmod 600 ~/.ssh/authorized_keys
- rsync -avz --exclude=.git --exclude=node_modules ./ ubuntu@ec2-16-170-254-93.eu-north-1.compute.amazonaws.com:/var/www/share
- ssh -o StrictHostKeyChecking=no ubuntu@ec2-16-170-254-93.eu-north-1.compute.amazonaws.com "cd /var/www/share && npm install && npm run build"
—
Let me know if you'd like more specific details on any of these optimizations!
### Steps to Add `bitbucket-pipelines.yml`:
1. **Navigate to the Root Directory of Your Repository**:
This is the main folder of your project, where your `README.md` and other core files are located.
2. **Create the `bitbucket-pipelines.yml` File**:
– You can create it manually in the root directory or through your IDE/text editor.
– The file should have the following structure:
# bitbucket-pipelines.yml
image: node:21
pipelines:
default:
- step:
name: Deploy Code to EC2
caches:
- node
script:
- echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$EC2_SSH_PUBLIC_KEY" > ~/.ssh/authorized_keys
- chmod 600 ~/.ssh/authorized_keys
- rsync -avz --exclude=.git --exclude=node_modules ./ ubuntu@ec2-16-170-254-93.eu-north-1.compute.amazonaws.com:/var/www/share
- ssh -o StrictHostKeyChecking=no ubuntu@ec2-16-170-254-93.eu-north-1.compute.amazonaws.com "cd /var/www/share && npm install && npm run build"
3. **Push the File to Bitbucket**:
Once the `bitbucket-pipelines.yml` file is added to your repository, commit and push it:
git add bitbucket-pipelines.yml
git commit -m "Add Bitbucket pipeline configuration"
git push origin main # or your branch
4. **Check the Pipeline**:
– Once you push the file, Bitbucket will automatically detect it and run the pipeline based on your configuration.
– You can monitor the progress and logs from the Bitbucket repository interface under **Pipelines**.
—
Let me know if you need further clarification!