Technical SEO and Structured Data

Why it matters: One wrong directive can remove a page from search; one misleading markup can lose eligibility for enhanced results. Choosing the right control for the goal is a high-leverage skill.

By the end you can:

• Choose between robots.txt, noindex, and canonicals
• Keep sitemaps and signals consistent
• Add accurate structured data
• Test and monitor

Learn (15 min)

1. Access vs. indexing (5 min)

• robots.txt asks compliant crawlers which paths they may request. It does not reliably remove an indexed URL.
• noindex (meta tag or header) asks search engines not to index a page, but the crawler must be able to fetch the page to see it. Blocking the page in robots.txt can prevent that.

<meta name="robots" content="noindex">

2. Canonicals and sitemaps (5 min)

A canonical names the preferred URL among duplicates. It's a signal, not a command.

<link rel="canonical" href="https://www.example.com/team-headshots/">

A sitemap lists URLs you consider important. Keep it accurate, and make internal links, redirects, canonicals, and sitemap entries agree on the preferred version.

3. Structured data (5 min)

Structured data is machine-readable markup that describes visible page content. Use a supported type only when it's accurate.

<script type="application/ld+json">
{ "@context": "https://schema.org",
  "@type": "LocalBusiness",
  "name": "Example Photo Studio",
  "address": { "@type": "PostalAddress", "addressLocality": "Saratoga Springs",
               "addressRegion": "NY" },
  "telephone": "+1-518-555-0100" }
</script>

Valid markup may make a page eligible for certain enhancements; it doesn't guarantee display or ranking. Test with Google's tools and review Search Console reports.

Apply (15 min)

Worked example (3 min). Same product page at two URLs, one with noindex, both in the sitemap. Fix: pick one canonical URL; 301-redirect or canonicalize the duplicate; remove the duplicate from the sitemap; make internal links point to the canonical; make sure robots.txt doesn't block it; inspect the rendered page and confirm index status.

Lab (7 min): Audit with a checklist.

1. Choose a site with at least two near-duplicate URLs (HTTP/HTTPS, www/non-www, trailing slash).
2. Record: what each URL returns (200/301/404), its canonical, and whether it's in the sitemap.
3. Write the correct configuration.
4. Draft a structured-data snippet for the page type and compare it with visible content.

Deliverable: A table of URLs, status, canonical, sitemap presence, and decision. You're done when: Every signal points to one preferred URL.

Check yourself (5 min)

1. What does robots.txt control?
2. What does noindex ask?
3. Is a canonical guaranteed?
4. Scenario: A page is "blocked by robots.txt" yet appears in results. Why can't noindex fix it?
5. What must structured data match?

Reference

Common mistakes

Cheat sheet: Right control → align canonical + links → maintain sitemap → mark up visible content → test and monitor.

Key terms: robots.txt, noindex, canonical, sitemap, structured data, JSON-LD.

Interview prompts: How do you handle duplicate URLs? When would you use noindex?

Go deeper: Technical requirements · Control what you share · Structured data policies · Search appearance

Next: Lesson 33 — answer-friendly content.