As an Infrastructure engineer, my terminal isn't just a shell. It's my control plane. Production debugging, Kubernetes firefighting, ASG scaling checks, S3 hotfixes, SSH pivots. Everything happens here. This post is generated from my actual ~/.zshrc on Arch Linux: what I really run, not a wishlist.
Shell is Zsh; theme is Powerlevel10k via Oh My Zsh. Plugins cover every tool I use daily: git, fzf, kubectl, helm, AWS, docker, terraform. Plus some quality-of-life utilities.
export ZSH="$HOME/.oh-my-zsh"
ZSH_THEME="powerlevel10k/powerlevel10k"
plugins=(
git
history-substring-search
fzf
kubectl
kubectx
helm
docker
docker-compose
terraform
aws
systemd
sudo
archlinux
ssh-agent
)
source $ZSH/oh-my-zsh.shThis setup has evolved over six years of continuous refinement, shaped by daily use across different environments and roles. Every plugin and shortcut exists because it made something faster, clearer, or more repeatable. The stack is intentionally simple and well-tested. Nothing flashy, just consistent.
For hardened or private environments, AWS Systems Manager is the default over port 22. SSH is kept only for legacy or old running setups.
ssm_fuzzy lists running EC2 instances in a region (default $AWS_DEFAULT_REGION, fallback ap-south-1), optionally filters by a query string, then uses fzf with a preview (InstanceId, Name, IP, AZ). Selecting one runs aws ssm start-session. Fallback without fzf: numbered list + read.
ssm_fuzzy() {
local REGION="${AWS_DEFAULT_REGION:-ap-south-1}"
local QUERY=""
for arg in "$@"; do
case "$arg" in
--region=*) REGION="${arg#*=}" ;;
*) QUERY="$arg" ;;
esac
done
local INSTANCES
INSTANCES=$(aws ec2 describe-instances \
--region "$REGION" \
--filters "Name=instance-state-name,Values=running" \
--query 'Reservations[].Instances[].{
ID: InstanceId,
Name: Tags[?Key==`Name`]|[0].Value,
IP: PrivateIpAddress,
AZ: Placement.AvailabilityZone
}' \
--output text)
[[ -z "$INSTANCES" ]] && echo "No running instances found." && return 1
[[ -n "$QUERY" ]] && INSTANCES=$(echo "$INSTANCES" | grep -i "$QUERY")
[[ -z "$INSTANCES" ]] && echo "No instances matched '$QUERY'" && return 1
local SELECTED=""
if command -v fzf >/dev/null && [[ -t 1 ]]; then
SELECTED=$(echo "$INSTANCES" | \
awk '{printf "%-20s %-30s %-15s %-10s\n", $1, $2, $3, $4}' | \
fzf --prompt="Select EC2 for SSM > " \
--preview "echo {} | awk '{print \"InstanceId: \" \$1 \"\nName: \" \$2 \"\nIP: \" \$3 \"\nAZ: \" \$4}'")
else
nl -w2 -s'. ' <<< "$INSTANCES"
read -p "Select number: " idx
SELECTED=$(echo "$INSTANCES" | sed -n "${idx}p")
fi
[[ -z "$SELECTED" ]] && return 0
local INSTANCE_ID
INSTANCE_ID=$(echo "$SELECTED" | awk '{print $1}')
echo "Starting SSM session to $INSTANCE_ID in $REGION"
aws ssm start-session --region "$REGION" --target "$INSTANCE_ID"
}SSH is only used where SSM isn't available. I don't type ssh -i ~/.ssh/<key-prod> ubuntu@10.x.x.x; I use sshprod 10.x.x.x. One key per environment, with a small helper that enforces the right key and default user so I can't accidentally hit prod with a dev key.
__ssh_with_key() {
local key="$1"
shift
if [[ "$1" == *"@"* ]]; then
ssh -i "$key" "$@"
else
ssh -i "$key" ubuntu@"$@"
fi
}
# Replace <key-dev>, <key-stag>, <key-prod>, <key-aws> with your key paths
sshdev() { __ssh_with_key ~/.ssh/<key-dev> "$@"; }
sshstag() { __ssh_with_key ~/.ssh/<key-stag> "$@"; }
sshprod() { __ssh_with_key ~/.ssh/<key-prod> "$@"; }
sshmum() { __ssh_with_key ~/.ssh/<key-aws> "$@"; }One command: incident. It creates a tmux session, splits panes, and starts k9s + kubectl + a shell. No manual pane setup during an incident.
incident() {
local SESSION="incident"
if tmux has-session -t "$SESSION" 2>/dev/null; then
tmux attach -t "$SESSION"
return
fi
tmux new-session -d -s "$SESSION" -n INCIDENT
tmux split-window -v -t "$SESSION"
tmux split-window -h -t "$SESSION":.1
tmux resize-pane -t "$SESSION":.0 -y 20
tmux send-keys -t $SESSION:.0 "k9s" C-m
tmux send-keys -t $SESSION:.1 "kubectl get pods" C-m
tmux send-keys -t $SESSION:.2 "echo 'Incident shell ready'" C-m
tmux select-pane -t $SESSION:.2
tmux attach -t $SESSION
}Layout: Top pane = k9s (overview). Bottom-left = pod list. Bottom-right = shell for commands. k9s helps me see what's happening; the shell is where I encode what needs to happen next and how I can find things at a more ground level.
These are custom helpers for debugging and cleanup, not in the default OMZ kubectl plugin. All accept -A / --all-namespaces where relevant.
kpn() {
kubectl get pods -o wide | awk '{print $1, $7}'
}kcrash() {
if [[ "$1" == "-A" ]] || [[ "$1" == "--all-namespaces" ]]; then
kubectl get pods -A -o wide | awk 'NR==1 || $5+0>0'
else
kubectl get pods -o wide | awk 'NR==1 || $4+0>0'
fi
}knotready() {
if [[ "$1" == "-A" ]] || [[ "$1" == "--all-namespaces" ]]; then
kubectl get pods -A -o wide | awk 'NR==1 || ($4!="Running" && $4!="Completed")'
else
kubectl get pods -o wide | awk 'NR==1 || ($3!="Running" && $3!="Completed")'
fi
}kbad() {
if [[ "$1" == "-A" ]] || [[ "$1" == "--all-namespaces" ]]; then
kubectl get pods -A -o wide | awk 'NR==1 || $4~/ImagePullBackOff|CrashLoopBackOff|Error|Pending|ErrImagePull/'
else
kubectl get pods -o wide | awk 'NR==1 || $3~/ImagePullBackOff|CrashLoopBackOff|Error|Pending|ErrImagePull/'
fi
}krestarts() {
local wide
if [[ "$1" == "-A" ]] || [[ "$1" == "--all-namespaces" ]]; then
wide=$(kubectl get pods -A -o wide)
echo "$wide" | head -1
echo "$wide" | tail -n +2 | sort -t' ' -k5 -rn
else
wide=$(kubectl get pods -o wide)
echo "$wide" | head -1
echo "$wide" | tail -n +2 | sort -t' ' -k4 -rn
fi
}klprev() {
kubectl logs "$1" --previous "${@:2}"
}kevict() {
if [[ "$1" == "-A" ]] || [[ "$1" == "--all-namespaces" ]]; then
kubectl get pods -A | awk 'NR>1 && $4~/Evicted/{print $1, $2}' | while read -r ns name; do kubectl delete pod -n "$ns" "$name"; done
else
kubectl get pods | awk 'NR>1 && $3~/Evicted/{print $1}' | while read -r name; do kubectl delete pod "$name"; done
fi
}No AWS console for routine lookups. These helpers cover the two most common patterns: finding instances by name tag, and inspecting ASG state, including a fuzzy picker with watch mode.
ec2_find() {
local NAME="$1"
local REGION="${AWS_DEFAULT_REGION:-ap-south-1}"
if [[ -z "$NAME" ]]; then
echo "Usage: ec2_find <name-substring> [region]"
return 1
fi
aws ec2 describe-instances \
--region "$REGION" \
--filters "Name=tag:Name,Values=*${NAME}*" \
--query 'Reservations[].Instances[].{
ID: InstanceId,
Name: Tags[?Key==`Name`]|[0].Value,
State: State.Name,
IP: PrivateIpAddress,
AZ: Placement.AvailabilityZone
}' \
--output table
}Region from $AWS_DEFAULT_REGION, fallback ap-south-1. Outputs a table with InstanceId, Name, PrivateIP, State, LaunchTime.
asg_instances() {
local ASG="$1"
local REGION="${2:-${AWS_DEFAULT_REGION:-ap-south-1}}"
if [[ -z "$ASG" ]]; then
echo "Usage: asg_instances <asg-name> [region]"
return 1
fi
aws ec2 describe-instances \
--region "$REGION" \
--filters "Name=tag:aws:autoscaling:groupName,Values=$ASG" \
--query 'Reservations[].Instances[].{
InstanceId: InstanceId,
Name: Tags[?Key==`Name`]|[0].Value,
PrivateIP: PrivateIpAddress,
State: State.Name,
LaunchTime: LaunchTime
}' \
--output table
}fzf over ASG names with a preview showing desired/min/max/instance count. --watch polls asg_instances every 5s and returns to the ASG list. Ctrl-C exits. Same region default as above.
asg_fuzzy() {
local REGION="${AWS_DEFAULT_REGION:-ap-south-1}"
local WATCH=""
for arg in "$@"; do
case "$arg" in
--watch) WATCH=1 ;;
--region=*) REGION="${arg#*=}" ;;
esac
done
while true; do
local ASG_LIST
ASG_LIST=$(aws autoscaling describe-auto-scaling-groups --region "$REGION" \
--query 'AutoScalingGroups[].AutoScalingGroupName' --output text | tr '\t' '\n')
[[ -z "$ASG_LIST" ]] && echo "No ASGs in $REGION" && return 1
local SELECTED
if command -v fzf >/dev/null && [[ -t 1 ]]; then
SELECTED=$(echo "$ASG_LIST" | fzf --prompt="ASG > " \
--preview "aws autoscaling describe-auto-scaling-groups --region $REGION --auto-scaling-group-names {} \
--query 'AutoScalingGroups[0].{Desired:DesiredCapacity,Min:MinSize,Max:MaxSize,Instances:length(Instances)}' --output text 2>/dev/null || echo {}")
else
echo "$ASG_LIST" | nl -w2 -s'. '
read -p "Select number: " idx
SELECTED=$(echo "$ASG_LIST" | sed -n "${idx}p")
fi
[[ -z "$SELECTED" ]] && return 0
asg_instances "$SELECTED" "$REGION"
[[ -z "$WATCH" ]] && return 0
echo "Refreshing in 5s (Ctrl-C to exit)..."
sleep 5
done
}s3edit downloads a file from S3 to a temp path, opens it in $EDITOR (vim fallback), then uploads it back when you save. No manual copy/paste.
s3edit() {
local S3PATH="$1"
if [[ -z "$S3PATH" ]]; then
echo "Usage: s3edit s3://bucket/key"
return 1
fi
if [[ "$S3PATH" != s3://* ]]; then
echo "Invalid S3 path. Must start with s3://"
return 1
fi
local TMPFILE
TMPFILE=$(mktemp /tmp/s3edit.XXXXXX)
echo "Downloading $S3PATH ..."
if ! aws s3 cp "$S3PATH" "$TMPFILE"; then
echo "Download failed."
rm -f "$TMPFILE"
return 1
fi
${EDITOR:-vim} "$TMPFILE"
echo "Reviewing changes before upload..."
aws s3 cp "$S3PATH" - | diff -u - "$TMPFILE"
read -q "REPLY?Upload these changes to $S3PATH? [y/N] "
echo
[[ "$REPLY" != "y" ]] && rm -f "$TMPFILE" && return 0
echo "Uploading back to $S3PATH ..."
aws s3 cp "$TMPFILE" "$S3PATH"
rm -f "$TMPFILE"
echo "Done."
}This is my setup, not a bootstrap script. Read every function, replace the placeholders, and test against a non-production account first.
sshprod chooses the production key, but a raw IP does not prove that the host is production. SSM with IAM and inventory is the safer default.
s3edit now shows a diff and asks before upload. It still cannot stop another writer from changing the object between download and upload. Use S3 versioning for anything that matters.
Cleanup helpers can delete resources. Keep the namespace and current Kubernetes context visible, and do not turn confirmation off just to save one keypress.
Every function and shortcut in this setup exists because it made something faster, clearer, or more repeatable. The five principles behind all of it:
incident) and crash detection (kcrash, kbad) mean the first 30 seconds of an incident aren't spent setting up windows.Beyond the shell itself, I rely on AI for a large part of my day-to-day work, from coding and refactoring to incident RCA, log exploration, and pattern detection during outages. I also use AI for metrics-driven analysis across databases and applications by connecting AI agents directly to the observability and APM stack from the terminal, turning raw telemetry into actionable insights much faster than manual dashboards ever could.
That said, this deserves a deep dive of its own: how AI fits into a DevOps workflow, how agents reason over metrics and traces, and how this changes incident response. I've intentionally kept that for a separate post.