Week 4 · Lab 7 — Smart Doc Analyzer

The architecture — what we're building

5 min

This is the pattern: upload triggers AI which writes results. The same shape powers customer support bots, expense tracking, content moderation, and fraud detection at major companies.

Upload PDF to S3 → Lambda fires → Calls Textract AND Comprehend → Saves combined results to DynamoDB.

You'll know it works when:

You upload a PDF to the S3 uploads/ folder
Within 10 seconds, a new row appears in your DynamoDB table
That row contains: the extracted text, the overall sentiment, and the named entities (people, places, organizations)

Heads up about Textract pricing

Textract is NOT free tier indefinitely — but you get 1,000 free pages per month for the first 3 months. More than enough for testing. Just don't process the entire library of Congress today.

Create the S3 bucket

10 min

Same drill as Week 2 Lab 4 — you've done this twice already.

S3 → Create bucket
Name: cloudwithshad-docs-<your-initials>-<random>
Region: same as everything else this bootcamp
Block all public access: keep ON (Lambda accesses it, not the public)
Click Create bucket
Open the bucket → create a folder named uploads/

Why a separate "uploads/" folder

It prevents accidental recursive triggers and keeps the bucket organized. We'll attach the Lambda trigger only to objects under uploads/.

Create the DynamoDB table

10 min

DynamoDB is AWS's serverless NoSQL database. Perfect for this — you don't manage a server, you don't manage schema, you just write JSON.

Navigate to DynamoDB → Tables → Create table
Table name: cloudwithshad-documents
Partition key: document_id (String)
Leave all other settings as defaults (On-demand capacity, no sort key)
Click Create table

That's it. Within 30 seconds the table is ready. No servers, no schema, no setup wizard for indexes.

Why DynamoDB beats RDS here

For this app, our data is unstructured (text + entities + sentiment scores) and we don't need joins. DynamoDB shines: it's serverless (no idle cost), scales automatically, and accepts JSON natively. RDS would be overkill.

Create the IAM role for Lambda

10 min

The Lambda function needs permissions to do five things: read from S3, call Textract, call Comprehend, write to DynamoDB, and write CloudWatch logs.

IAM → Roles → Create role
Trusted entity: AWS service
Use case: Lambda
Attach these policies (search and tick each):
- AWSLambdaBasicExecutionRole
- AmazonS3ReadOnlyAccess
- AmazonTextractFullAccess
- ComprehendFullAccess
- AmazonDynamoDBFullAccess
Role name: cloudwithshad-docs-role
Create role

Production note

FullAccess policies are too broad for production. In real systems you'd create custom policies scoped to ONLY the specific table/bucket/service. For learning, FullAccess is fine.

Create the Lambda function

20 min

Lambda → Create function
Author from scratch
Function name: cloudwithshad-doc-analyzer
Runtime: Python 3.12
Architecture: x86_64
Execution role: Use existing → cloudwithshad-docs-role
Click Create function

Once created, scroll to the Code tab and paste this:

Option A · Managed AI (Amazon Comprehend)

This version uses Amazon Comprehend for sentiment + entities. It's the most accurate, but Comprehend is not on the perpetual free tier — depending on your account it may need billing enabled and can cost a dollar or two. If you'd rather not upgrade your account, use Option B just below — it does the same job for free.

import boto3
import json
import uuid
from datetime import datetime

s3        = boto3.client('s3')
textract  = boto3.client('textract')
comprehend = boto3.client('comprehend')
dynamo    = boto3.resource('dynamodb')
table     = dynamo.Table('cloudwithshad-documents')

def lambda_handler(event, context):
    # 1. Grab the uploaded object info from the S3 event
    bucket = event['Records'][0]['s3']['bucket']['name']
    key    = event['Records'][0]['s3']['object']['key']

    # Only process files in the uploads/ folder
    if not key.startswith('uploads/'):
        return {'status': 'skipped'}

    # 2. Call Textract to extract text from the PDF/image
    textract_response = textract.detect_document_text(
        Document={'S3Object': {'Bucket': bucket, 'Name': key}}
    )

    # Join all the detected words into one string
    extracted_text = ' '.join([
        block['Text'] for block in textract_response['Blocks']
        if block['BlockType'] == 'LINE'
    ])

    # Comprehend has a 5000-byte limit per call
    text_for_analysis = extracted_text[:5000]

    # 3. Call Comprehend for sentiment
    sentiment_result = comprehend.detect_sentiment(
        Text=text_for_analysis, LanguageCode='en'
    )

    # 4. Call Comprehend for entities
    entities_result = comprehend.detect_entities(
        Text=text_for_analysis, LanguageCode='en'
    )

    entities = [
        {'text': e['Text'], 'type': e['Type'], 'score': str(e['Score'])}
        for e in entities_result['Entities'][:10]   # top 10 entities
    ]

    # 5. Save everything to DynamoDB
    doc_id = str(uuid.uuid4())
    table.put_item(Item={
        'document_id': doc_id,
        'filename': key,
        'processed_at': datetime.utcnow().isoformat(),
        'extracted_text': extracted_text[:1000],  # preview
        'sentiment': sentiment_result['Sentiment'],
        'sentiment_scores': {
            k: str(v) for k, v in sentiment_result['SentimentScore'].items()
        },
        'entities': entities,
    })

    return {
        'statusCode': 200,
        'document_id': doc_id,
        'sentiment': sentiment_result['Sentiment'],
        'entity_count': len(entities)
    }
      

Option B · Free, no account upgrade

Don't want to enable billing or pay for Comprehend? Use this version instead. It keeps Textract (which has a free 1,000 pages/month window) but does the sentiment + entity analysis in plain Python inside the same Lambda — so it costs nothing beyond the Lambda free tier. Same architecture, same DynamoDB result, $0.

Paste this into your Lambda's Code tab instead of the Option A code:

import boto3
import uuid
import re
from datetime import datetime

s3       = boto3.client('s3')
textract = boto3.client('textract')
dynamo   = boto3.resource('dynamodb')
table    = dynamo.Table('cloudwithshad-documents')

# --- Tiny built-in sentiment word lists (no paid AI service needed) ---
POSITIVE = {'good','great','excellent','positive','happy','love','best','strong','success','improve','benefit','effective','reliable','recommend','pleased','satisfied','achieve','gain','advantage','opportunity','wonderful','amazing'}
NEGATIVE = {'bad','poor','terrible','negative','hate','worst','awful','fail','failure','weak','problem','issue','difficult','delay','risk','loss','concern','disappointed','unable','error','broken','decline','complaint'}

def analyze_sentiment(text):
    words = re.findall(r"[a-z']+", text.lower())
    pos = sum(1 for w in words if w in POSITIVE)
    neg = sum(1 for w in words if w in NEGATIVE)
    total = pos + neg
    if total == 0:
        return 'NEUTRAL', {'Positive':'0.0', 'Negative':'0.0', 'Neutral':'1.0', 'Mixed':'0.0'}
    p = round(pos / total, 3)
    n = round(neg / total, 3)
    if pos > 0 and neg > 0 and abs(pos - neg) <= 1:
        label = 'MIXED'
    elif pos > neg:
        label = 'POSITIVE'
    elif neg > pos:
        label = 'NEGATIVE'
    else:
        label = 'NEUTRAL'
    scores = {'Positive': str(p), 'Negative': str(n),
              'Neutral': str(round(max(0.0, 1 - p - n), 3)), 'Mixed': '0.0'}
    return label, scores

def extract_entities(text):
    # Capitalized words/phrases = likely names, orgs, places
    candidates = re.findall(r'\b([A-Z][a-z]+(?:\s+[A-Z][a-z]+){0,2})\b', text)
    emails = re.findall(r'\b[\w.\-]+@[\w.\-]+\.\w+\b', text)
    seen, entities = set(), []
    for c in candidates:
        if c.lower() in seen or len(c) < 3:
            continue
        seen.add(c.lower())
        entities.append({'text': c, 'type': 'PROPER_NOUN', 'score': '0.80'})
        if len(entities) >= 10:
            break
    for e in emails[:3]:
        entities.append({'text': e, 'type': 'EMAIL', 'score': '0.99'})
    return entities[:10]

def lambda_handler(event, context):
    bucket = event['Records'][0]['s3']['bucket']['name']
    key    = event['Records'][0]['s3']['object']['key']
    if not key.startswith('uploads/'):
        return {'status': 'skipped'}

    # Textract still extracts the text (free 1,000 pages/month window)
    textract_response = textract.detect_document_text(
        Document={'S3Object': {'Bucket': bucket, 'Name': key}}
    )
    extracted_text = ' '.join([
        b['Text'] for b in textract_response['Blocks']
        if b['BlockType'] == 'LINE'
    ])

    # Analyze IN PYTHON — no Comprehend, no extra cost
    sentiment, sentiment_scores = analyze_sentiment(extracted_text)
    entities = extract_entities(extracted_text)

    doc_id = str(uuid.uuid4())
    table.put_item(Item={
        'document_id': doc_id,
        'filename': key,
        'processed_at': datetime.utcnow().isoformat(),
        'extracted_text': extracted_text[:1000],
        'sentiment': sentiment,
        'sentiment_scores': sentiment_scores,
        'entities': entities,
        'analysis_method': 'python-builtin'
    })

    return {
        'statusCode': 200,
        'document_id': doc_id,
        'sentiment': sentiment,
        'entity_count': len(entities)
    }
      

Two small differences for Option B

1. Skip the Comprehend permission. In the IAM role step earlier, you can leave off ComprehendFullAccess — Option B never calls Comprehend. You only need S3 read, Textract, DynamoDB, and CloudWatch logs.

2. Same 60s timeout is fine. Only Textract makes a network call now; the Python analysis is instant.

Honest trade-off (worth understanding)

Option B's analysis is simpler than Comprehend's machine-learning models — it uses word lists for sentiment and capitalisation for entities. It's perfect for learning the full pipeline at zero cost, and it's a great lesson in why managed AI services exist: Comprehend is more accurate because it's trained on huge datasets. When you're ready to spend a little (or in the AI course), swapping Option B for Comprehend — or for a Bedrock model — is a one-function change. The architecture you built stays exactly the same.

Click Deploy.

Bump the timeout

Configuration → General configuration → Edit. Change timeout from 3 seconds to 60 seconds. Textract + Comprehend calls can take 5-15s on bigger documents.

Now add the S3 trigger:

In the Lambda function page, click "+ Add trigger"
Select S3
Bucket: your cloudwithshad-docs-... bucket
Event type: All object create events
Prefix: uploads/
Tick the recursive acknowledgement
Click Add

Test the pipeline

15 min

The moment of truth.

Find a real document to test with. Good options:

A receipt photo from your phone
A screenshot of an article or news page
A simple PDF (under 5 MB to start)
Your CV (works great — lots of entities to detect!)

Open S3 → your bucket → uploads/
Click Upload and drop in the file
Wait 10-15 seconds
Open DynamoDB → Tables → cloudwithshad-documents → Explore items
Click Scan
You should see a new row with the extracted text, sentiment, and entities!

Take a screenshot

Capture the DynamoDB item showing the sentiment + entities pulled from your document. Post it in the cohort group with the caption "Lab 7 done — AI just read my CV!" 🎉

Debugging when it doesn't work

CloudWatch → Log groups → /aws/lambda/cloudwithshad-doc-analyzer. The error will tell you everything. Most common: missing permissions on the IAM role, or the Lambda timing out (bump timeout to 60s).

Clean up

5 min

⚠ Cleanup checklist

S3 bucket: empty it, then delete
Lambda function: Lambda → cloudwithshad-doc-analyzer → Delete
DynamoDB table: DynamoDB → Tables → cloudwithshad-documents → Delete
IAM role: IAM → Roles → cloudwithshad-docs-role → Delete
CloudWatch log group: /aws/lambda/cloudwithshad-doc-analyzer → Delete (optional)

A note on Textract free tier

The Textract free tier is generous — 1,000 pages/month for 3 months. After that it's ~$1.50 per 1,000 pages. You won't see surprise bills unless you process thousands of documents. DynamoDB is free up to 25 GB of storage.

🧠

Lab 7 complete!

You just chained 4 AWS services to build a real AI pipeline. This is the foundation of every modern "smart document" tool out there.

Next: Lab 8 — Your AI Capstone Project

Build a Smart Doc Analyzer with AI

⚠ Cleanup checklist

Lab 7 complete!