What is AWS Console API Call Architecture?

When you use the AWS Management Console in your browser, it makes API calls to AWS services. Understanding how these calls work is important for debugging, blocking specific services, or understanding the security model.

Two Types of API Calls

1. Direct Browser → AWS Service API

Example: SSM (Systems Manager) API call

Browser JavaScript
    ↓
https://ssm.us-east-1.amazonaws.com/
    ↓
AWS Service responds directly

Characteristics:

  • Browser makes HTTPS request directly to service endpoint
  • Uses AWS SigV4 authentication (signed with temporary credentials)
  • Subject to CORS (Cross-Origin Resource Sharing) rules
  • Can be blocked via /etc/hosts
  • Shows in DevTools Network tab with service endpoint URL

Example from Session Manager:

{
  "method": "POST",
  "url": "https://ssm.us-east-1.amazonaws.com/",
  "headers": {
    "x-amz-target": "AmazonSSM.DescribeInstanceInformation",
    "authorization": "AWS4-HMAC-SHA256 Credential=...",
    "x-amz-security-token": "IQoJb3JpZ2luX2VjEO..."
  }
}

2. Browser → Console Backend → AWS Service API

Example: Telemetry, Panorama (analytics)

Browser JavaScript
    ↓
https://us-east-1.prod.pl.panorama.console.api.aws/
    ↓
Console Backend (AWS-managed proxy)
    ↓
AWS Service

Characteristics:

  • Browser talks to Console backend, not service directly
  • Console backend makes actual AWS API calls
  • Cannot be blocked by blocking service endpoints
  • sourceIPAddress in CloudTrail shows Console backend IP, not your IP
  • Used for analytics, telemetry, cross-service features

Key Technical Concepts

DNS Resolution: /etc/hosts vs dig

Important: /etc/hosts affects applications (browsers, curl, ping), but NOT dig.

# dig queries DNS servers directly, ignores /etc/hosts
dig +short ssm.us-east-1.amazonaws.com
44.216.202.52  # Returns real IP even with /etc/hosts block

# Browser/curl use system resolver, checks /etc/hosts first
curl https://ssm.us-east-1.amazonaws.com/
# Connection refused (uses 127.0.0.1 from /etc/hosts)

Why dig ignores /etc/hosts:

dig is a DNS debugging tool that queries DNS servers directly:

dig command
    ↓
Bypasses system resolver
    ↓
Queries DNS server directly (8.8.8.8, 1.1.1.1, etc.)
    ↓
Returns real DNS records

Browser/applications use system resolver:

Browser/curl/ping
    ↓
Uses system resolver (getaddrinfo())
    ↓
Checks /etc/hosts FIRST
    ↓
If found: Use /etc/hosts entry
If not found: Query DNS server

To test /etc/hosts blocking:

# These commands respect /etc/hosts
ping ssm.us-east-1.amazonaws.com
# PING ssm.us-east-1.amazonaws.com (127.0.0.1)

curl -I https://ssm.us-east-1.amazonaws.com/
# curl: (7) Failed to connect to ssm.us-east-1.amazonaws.com port 443

# This command ignores /etc/hosts
dig +short ssm.us-east-1.amazonaws.com
# 44.216.202.52 (real AWS IP)

IPv4 vs IPv6 Resolution

AWS service endpoints support dual-stack (both IPv4 and IPv6):

# Query IPv4 address (A record)
dig A ssm.us-east-1.amazonaws.com
# Returns: 44.216.202.52

# Query IPv6 address (AAAA record)
dig AAAA ssm.us-east-1.amazonaws.com
# Returns: 2600:1f18:... (IPv6 address)

Modern browsers prefer IPv6 when available. If only one protocol is blocked in /etc/hosts, browser falls back to real DNS.

Happy Eyeballs (RFC 8305)

Browser algorithm for choosing IPv4 vs IPv6:

DNS returns both A (IPv4) and AAAA (IPv6) records
    ↓
Browser checks: "Do I have IPv6 connectivity?"
    ↓
YES → Try IPv6 first
NO → Use IPv4
    ↓
If one protocol fails → Try other protocol (DNS fallback)
If BOTH protocols fail → Report error (no fallback)

How to Block AWS Service API Calls via /etc/hosts

Add Both IPv4 and IPv6 Entries

# Add both entries to /etc/hosts
sudo bash -c 'cat >> /etc/hosts << EOF
127.0.0.1       ssm.us-east-1.amazonaws.com
::1             ssm.us-east-1.amazonaws.com
EOF'

Result: Browser tries both protocols, both fail, no DNS fallback occurs.

Why Both Entries Are Required

With only IPv4 (127.0.0.1):

Browser checks /etc/hosts
    ↓
Finds: 127.0.0.1 ssm.us-east-1.amazonaws.com
    ↓
Browser prefers IPv6 (system default)
    ↓
Ignores IPv4-only entry
    ↓
Uses real DNS to get IPv6 address
    ↓
API call succeeds

With only IPv6 (::1):

Browser checks /etc/hosts
    ↓
Finds: ::1 ssm.us-east-1.amazonaws.com
    ↓
Tries IPv6 (::1:443) - connection refused
    ↓
Browser thinks: "Maybe wrong protocol"
    ↓
Falls back to real DNS for IPv4
    ↓
API call succeeds

With BOTH IPv4 and IPv6:

Browser checks /etc/hosts
    ↓
Finds both: 127.0.0.1 and ::1
    ↓
Tries IPv6 first (::1:443) - connection refused
    ↓
Tries IPv4 next (127.0.0.1:443) - connection refused
    ↓
BOTH protocols failed
    ↓
Browser thinks: "This domain is intentionally blocked"
    ↓
NO DNS fallback happens
    ↓
API call fails ✓

Technical Explanation

Happy Eyeballs Fallback Logic:

  • If one protocol fails → Assume network issue, try other protocol via real DNS
  • If both protocols fail → Assume intentional block, no fallback

Key Point: Browsers only skip DNS fallback when they’ve exhausted all available protocols from /etc/hosts.

Real-World Example: Session Manager

When you click “Session Manager” tab in EC2 Console:

1. Browser makes direct API call:

// JavaScript in browser
POST https://ssm.us-east-1.amazonaws.com/
{
  "Filters": [{
    "Key": "InstanceIds",
    "Values": ["i-04d5071d51847a388"]
  }]
}

2. With both IPv4 and IPv6 blocked (ERR_CONNECTION_REFUSED):

{
  "response": {
    "status": 0,
    "_error": "net::ERR_CONNECTION_REFUSED"
  },
  "timings": {
    "blocked": 5.287
  }
}

3. Console shows error:

  • Red flashbar: “Failed to describe instance information”
  • Session Manager tab shows error state

Technical Terms

  • CORS (Cross-Origin Resource Sharing): Browser security that controls which domains can make API calls to other domains
  • SigV4 (AWS Signature Version 4): AWS authentication method that signs API requests with access keys
  • Dual-stack: Network/service supporting both IPv4 and IPv6
  • A record: DNS record mapping domain to IPv4 address
  • AAAA record: DNS record mapping domain to IPv6 address (quad-A because IPv6 is 4x larger)
  • Happy Eyeballs: RFC 8305 algorithm for choosing between IPv4 and IPv6
  • DNS fallback: Browser retry mechanism when one protocol fails
  • 127.0.0.1: IPv4 localhost (loopback address)
  • ::1: IPv6 localhost (loopback address)
  • Connection refused: TCP error when nothing is listening on IP:port
  • /etc/hosts: Local DNS override file, checked by system resolver (not by dig)
  • dig: DNS debugging tool that queries DNS servers directly, bypasses /etc/hosts
  • System resolver: OS-level DNS resolution (getaddrinfo()) that checks /etc/hosts first
  • getaddrinfo(): System function that resolves domain names, respects /etc/hosts

Debugging Tips

Check what browser is actually connecting to:

# Browser DevTools → Network tab
# Look at "Remote Address" column
# Example: [2600:1f18:...]:443 = IPv6
# Example: 52.46.128.5:443 = IPv4

Verify DNS resolution (respects /etc/hosts):

# Use ping (respects /etc/hosts)
ping -c 1 ssm.us-east-1.amazonaws.com

# Use curl (respects /etc/hosts)
curl -I https://ssm.us-east-1.amazonaws.com/

# Use getent (respects /etc/hosts)
getent hosts ssm.us-east-1.amazonaws.com

# macOS alternative
dscacheutil -q host -a name ssm.us-east-1.amazonaws.com

Query real DNS (ignores /etc/hosts):

# dig always queries DNS directly
dig +short ssm.us-east-1.amazonaws.com

# Force IPv4
dig -4 ssm.us-east-1.amazonaws.com

# Force IPv6
dig -6 ssm.us-east-1.amazonaws.com

Check active connections:

# See what's listening on port 443
sudo lsof -i :443

# See established connections
sudo lsof -i :443 | grep ESTABLISHED

Key Takeaways

  1. Two architectures: Direct browser→service vs browser→console backend→service
  2. DNS matters: Browser uses local /etc/hosts before real DNS
  3. dig ignores /etc/hosts: It’s a DNS debugging tool that queries DNS servers directly
  4. Dual-stack blocking: Must block both IPv4 and IPv6 to prevent fallback
  5. Happy Eyeballs: Browser tries both protocols, falls back if only one fails
  6. CloudTrail limitation: Can’t determine exact endpoint from eventSource alone
  7. Console backend: Some API calls go through AWS proxy, can’t be blocked by endpoint blocking

Notes

  • Console backend calls show AWS IP in CloudTrail sourceIPAddress, not your IP
  • eventSource in CloudTrail (e.g., ssm.amazonaws.com) is service identifier, not endpoint URL
  • Actual endpoint is service.region.amazonaws.com (combine eventSource + awsRegion)
  • /etc/hosts is checked before real DNS queries by system resolver (getaddrinfo)
  • dig, nslookup, host commands bypass /etc/hosts for DNS debugging purposes
  • Modern macOS/Linux systems prefer IPv6 when available