Most important things we need to improve on:
Most important idea currently:
a pool with extremely high sensitivity, a few irrelevant items sorted out, then “the full picture” for users who want it all: only original title, link, original date, EMFS date, source name → from that pool we select the stories we highlight → double-curated news; our selections, like a newspaper and much like our current site. So we should aim to also make the intermediate “give me all” version.
Elements are listed roughly in the same order as a story draft YAML/frontmatter file. “Shown” means visible to readers somewhere on the story, cards, source box, or share tools.
| YAML field / element | Required? | Shown to reader? | Meaning / editorial use |
|---|---|---|---|
title | Yes | Yes | Main headline. Should be specific, credible, and shareable — not tabloid. |
date | Yes | Yes | Original source publication date. |
emfs_date | Yes | Yes | Date added to EMF Signal. |
emfs_date | Yes | No | Date added to EMF Signal (ISO format, used for sorting and display). |
source | Yes | Yes | Publisher/outlet name. |
source_type | Yes | Yes | Type label: Press, Research, Voice, NGO, Mainstream, Therapeutic, Industry, etc. |
source_language | Yes | Yes | Original source language (ISO 639-1, e.g. "en", "fr", "de"). Required — controls whether original-language quotes are shown beneath translations. |
source_url | Yes | Yes | Link to the original source. |
source_title | Yes | Yes | Original source title shown in the source box. |
source_domain | Yes | Yes | Source domain shown with the source link. |
icon | Optional | Yes | Emoji/icon before the title, often country or media type. |
summary | Yes | Yes | EMF Signal’s main summary: text above quotes and card/list text. Usually 2–3 sentences. |
discovery_source | Recommended | On drafts page | How the draft was found or created; helps editor trust the pipeline. |
share_text | Recommended | Share tools | Social media text for X/Bluesky. Separate from summary; always English. |
piece_has_audio/video | Optional | Yes, if set | Marks audio/video attached to the EMF Signal piece itself. |
source_has_audio/video | Optional | Yes, if set | Marks audio/video attached to the original source. |
source_note | Optional | Yes, if set | Info box below summary explaining source context (e.g. transcript origin, translation note). Supports markdown links. |
tip_credit | Optional | Yes, if set | Short H/T or courtesy line for discovery credit. Shows above the Source box, not as a quote or info box. Supports markdown links. |
change_note | Optional | Yes, if set | Structured update/history note shown below source note. |
categories | Recommended | Yes | Topic/category tags and site placement. |
study | Optional | Yes, if set | Underlying research box. Only for real studies with DOI/PMID/credible reference. |
event | Optional | Sometimes | Links the piece to a recurring news event/topic cluster. |
quotes | Expected | Yes | Verified source excerpts supporting the title and summary. |
Markdown after --- | Optional | Yes | Free-form story content. Use sparingly; for recurring notes/transcripts, prefer structured template fields. |
A simplified dependency view: what type of node each step is, which programs are involved, what each node depends on, and what it feeds into.
| Tool | Script | Status | Scanned | New | Time |
|---|---|---|---|---|---|
| EMF Portal Literature | emf-portal.py | ok | - | 0 | 3.1s |
| Brave Search (core) | brave-search.py | ok | 3 | 0 | 41.9s |
| Brave Search (policy) | brave-search.py | ok | 9 | 0 | 24.4s |
| Brave Search (emerging) | brave-search.py | ok | 4 | 0 | 21.5s |
| Brave Search (cancer) | brave-search.py | ok | 1 | 0 | 23.5s |
| Semantic Scholar | semantic-scholar.py | ok | - | 0 | 29.5s |
| Preprints (Europe PMC) | search_medrxiv.py | ok | - | 0 | 0.7s |
| YouTube Influencers | yt-transcript-scanner.py | ok | - | 0 | 5.0s |
| Bluesky | bluesky-search.py | ok | - | 0 | 0.6s |
| Org Scanner | org-scanner.py | ok | 4 | 8 | 11.4s |
| Voice Scanner | voice-scanner.py | ok | - | 0 | 1.3s |
| Newsletter Fetcher | newsletter-fetcher.py | ok | - | 2 | 8.9s |
| DSA Norway | ../aggregator/dsa_scraper.py | ok | - | 0 | 0.5s |
| SSM Sweden | ../aggregator/ssm_scraper.py | ok | - | 0 | 0.5s |
| ARPANSA Australia | ../aggregator/arpansa_scraper.py | error | - | 0 | 31.1s |
| STUK Finland | ../aggregator/stuk_scraper.py | ok | - | 0 | 1.3s |
| RIVM Netherlands | ../aggregator/rivm_scraper.py | ok | - | 0 | 0.5s |
| BAFU Switzerland | ../aggregator/bafu_scraper.py | ok | - | 0 | 1.8s |
| Israel MoEP | ../aggregator/israel_moep_scraper.py | ok | - | 0 | 1.2s |
| Site Patrol | site-patrol.py | ok | 19 | 0 | 23.7s |
| Tool | Script | Status | Scanned | New | Time |
|---|---|---|---|---|---|
| EMF Portal Literature | emf-portal.py | ok | - | 0 | 3.1s |
| Brave Search (core) | brave-search.py | ok | 3 | 0 | 41.9s |
| Brave Search (policy) | brave-search.py | ok | 9 | 0 | 24.4s |
| Brave Search (emerging) | brave-search.py | ok | 4 | 0 | 21.5s |
| Brave Search (cancer) | brave-search.py | ok | 1 | 0 | 23.5s |
| Semantic Scholar | semantic-scholar.py | ok | - | 0 | 29.5s |
| Preprints (Europe PMC) | search_medrxiv.py | ok | - | 0 | 0.7s |
| YouTube Influencers | yt-transcript-scanner.py | ok | - | 0 | 5.0s |
| Bluesky | bluesky-search.py | ok | - | 0 | 0.6s |
| Org Scanner | org-scanner.py | ok | 4 | 8 | 11.4s |
| Voice Scanner | voice-scanner.py | ok | - | 0 | 1.3s |
| Newsletter Fetcher | newsletter-fetcher.py | ok | - | 2 | 8.9s |
| DSA Norway | ../aggregator/dsa_scraper.py | ok | - | 0 | 0.5s |
| SSM Sweden | ../aggregator/ssm_scraper.py | ok | - | 0 | 0.5s |
| ARPANSA Australia | ../aggregator/arpansa_scraper.py | error | - | 0 | 31.1s |
| STUK Finland | ../aggregator/stuk_scraper.py | ok | - | 0 | 1.3s |
| RIVM Netherlands | ../aggregator/rivm_scraper.py | ok | - | 0 | 0.5s |
| BAFU Switzerland | ../aggregator/bafu_scraper.py | ok | - | 0 | 1.8s |
| Israel MoEP | ../aggregator/israel_moep_scraper.py | ok | - | 0 | 1.2s |
| Site Patrol | site-patrol.py | ok | 19 | 0 | 23.7s |
| Tool | Script | Status | Scanned | New | Time |
|---|---|---|---|---|---|
| RSS Feeds | check-feeds.py | ok | 322 | 0 | 7.4s |
| PubMed | pubmed-search.py | ok | 25 | 25 | 4.4s |
| EMF Portal | emf-portal.py | ok | 10 | 10 | 0.3s |
| Tool | Script | Status | Scanned | New | Time |
|---|---|---|---|---|---|
| RSS Feeds | check-feeds.py | ok | 322 | 0 | 10.8s |
| PubMed | pubmed-search.py | ok | 25 | 25 | 4.0s |
| EMF Portal | emf-portal.py | ok | 10 | 10 | 0.3s |
| Tool | Script | Status | Scanned | New | Time |
|---|---|---|---|---|---|
| EMF Portal Literature | emf-portal.py | ok | - | 0 | 2.5s |
| Brave Search (core) | brave-search.py | ok | 2 | 0 | 43.2s |
| Brave Search (policy) | brave-search.py | ok | 2 | 0 | 25.8s |
| Brave Search (emerging) | brave-search.py | ok | 3 | 0 | 23.2s |
| Brave Search (cancer) | brave-search.py | ok | 3 | 0 | 21.0s |
| Semantic Scholar | semantic-scholar.py | ok | - | 0 | 32.2s |
| Preprints (Europe PMC) | search_medrxiv.py | ok | - | 0 | 0.8s |
| YouTube Influencers | yt-transcript-scanner.py | ok | - | 0 | 4.5s |
| Bluesky | bluesky-search.py | ok | - | 0 | 0.5s |
| Org Scanner | org-scanner.py | ok | 5 | 10 | 19.0s |
| Voice Scanner | voice-scanner.py | ok | - | 0 | 1.2s |
| Newsletter Fetcher | newsletter-fetcher.py | ok | - | 3 | 8.0s |
| DSA Norway | ../aggregator/dsa_scraper.py | ok | - | 0 | 0.6s |
| SSM Sweden | ../aggregator/ssm_scraper.py | ok | - | 0 | 0.7s |
| ARPANSA Australia | ../aggregator/arpansa_scraper.py | error | - | 0 | 32.3s |
| STUK Finland | ../aggregator/stuk_scraper.py | ok | - | 0 | 1.3s |
| RIVM Netherlands | ../aggregator/rivm_scraper.py | ok | - | 0 | 2.0s |
| BAFU Switzerland | ../aggregator/bafu_scraper.py | ok | - | 0 | 3.4s |
| Israel MoEP | ../aggregator/israel_moep_scraper.py | ok | - | 0 | 0.4s |
| Site Patrol | site-patrol.py | ok | 19 | 0 | 18.2s |
| Tool | Script | Status | Scanned | New | Time |
|---|---|---|---|---|---|
| RSS Feeds | check-feeds.py | ok | 320 | 0 | 14.9s |
| PubMed | pubmed-search.py | ok | 25 | 25 | 3.9s |
| EMF Portal | emf-portal.py | ok | 10 | 10 | 0.3s |
| Tool | Script | Status | Scanned | New | Time |
|---|---|---|---|---|---|
| EMF Portal Literature | emf-portal.py | ok | - | 0 | 4.1s |
| Brave Search (core) | brave-search.py | ok | 8 | 0 | 42.5s |
| Brave Search (policy) | brave-search.py | ok | 1 | 0 | 27.7s |
| Brave Search (emerging) | brave-search.py | ok | 2 | 0 | 21.1s |
| Brave Search (cancer) | brave-search.py | ok | - | 0 | 22.6s |
| Semantic Scholar | semantic-scholar.py | ok | - | 0 | 26.7s |
| Preprints (Europe PMC) | search_medrxiv.py | ok | - | 0 | 0.8s |
| YouTube Influencers | yt-transcript-scanner.py | ok | 1 | 0 | 4.5s |
| Bluesky | bluesky-search.py | ok | - | 0 | 0.6s |
| Org Scanner | org-scanner.py | ok | 3 | 6 | 11.0s |
| Voice Scanner | voice-scanner.py | ok | - | 0 | 1.1s |
| Newsletter Fetcher | newsletter-fetcher.py | ok | - | 3 | 10.5s |
| DSA Norway | ../aggregator/dsa_scraper.py | ok | - | 0 | 0.5s |
| SSM Sweden | ../aggregator/ssm_scraper.py | ok | - | 0 | 0.8s |
| ARPANSA Australia | ../aggregator/arpansa_scraper.py | error | - | 0 | 31.6s |
| STUK Finland | ../aggregator/stuk_scraper.py | ok | - | 0 | 1.2s |
| RIVM Netherlands | ../aggregator/rivm_scraper.py | ok | - | 0 | 2.0s |
| BAFU Switzerland | ../aggregator/bafu_scraper.py | ok | - | 0 | 1.7s |
| Israel MoEP | ../aggregator/israel_moep_scraper.py | ok | - | 0 | 0.3s |
| Site Patrol | site-patrol.py | ok | 19 | 0 | 17.3s |
| Tool | Script | Status | Scanned | New | Time |
|---|---|---|---|---|---|
| RSS Feeds | check-feeds.py | ok | 319 | 0 | 9.5s |
| PubMed | pubmed-search.py | ok | 23 | 23 | 3.6s |
| EMF Portal | emf-portal.py | ok | 10 | 10 | 0.4s |
| Tool | Script | Status | Scanned | New | Time |
|---|---|---|---|---|---|
| EMF Portal Literature | emf-portal.py | ok | - | 0 | 2.8s |
| Brave Search (core) | brave-search.py | ok | 6 | 0 | 37.3s |
| Brave Search (policy) | brave-search.py | ok | 1 | 0 | 24.4s |
| Brave Search (emerging) | brave-search.py | ok | 2 | 0 | 21.8s |
| Brave Search (cancer) | brave-search.py | ok | - | 0 | 21.6s |
| Semantic Scholar | semantic-scholar.py | ok | - | 0 | 29.4s |
| Preprints (Europe PMC) | search_medrxiv.py | ok | - | 0 | 0.7s |
| YouTube Influencers | yt-transcript-scanner.py | ok | - | 0 | 2.3s |
| Bluesky | bluesky-search.py | ok | - | 0 | 0.6s |
| Org Scanner | org-scanner.py | ok | 6 | 12 | 9.6s |
| Voice Scanner | voice-scanner.py | ok | - | 0 | 1.3s |
| Newsletter Fetcher | newsletter-fetcher.py | ok | - | 0 | 7.9s |
| DSA Norway | ../aggregator/dsa_scraper.py | ok | - | 0 | 0.5s |
| SSM Sweden | ../aggregator/ssm_scraper.py | ok | - | 0 | 0.5s |
| ARPANSA Australia | ../aggregator/arpansa_scraper.py | error | - | 0 | 30.4s |
| STUK Finland | ../aggregator/stuk_scraper.py | ok | - | 0 | 0.9s |
| RIVM Netherlands | ../aggregator/rivm_scraper.py | ok | - | 0 | 2.0s |
| BAFU Switzerland | ../aggregator/bafu_scraper.py | ok | - | 0 | 1.3s |
| Israel MoEP | ../aggregator/israel_moep_scraper.py | ok | - | 0 | 0.2s |
| Site Patrol | site-patrol.py | ok | 19 | 0 | 16.1s |
| Tool | Script | Status | Scanned | New | Time |
|---|---|---|---|---|---|
| RSS Feeds | check-feeds.py | ok | 319 | 0 | 12.1s |
| PubMed | pubmed-search.py | ok | 20 | 20 | 2.6s |
| EMF Portal | emf-portal.py | ok | 10 | 10 | 0.4s |
| Tool | Script | Status | Scanned | New | Time |
|---|---|---|---|---|---|
| EMF Portal Literature | emf-portal.py | ok | - | 0 | 2.4s |
| Brave Search (core) | brave-search.py | ok | 2 | 0 | 44.8s |
| Brave Search (policy) | brave-search.py | ok | 3 | 0 | 26.3s |
| Brave Search (emerging) | brave-search.py | ok | 3 | 0 | 22.9s |
| Brave Search (cancer) | brave-search.py | ok | 1 | 0 | 22.5s |
| Semantic Scholar | semantic-scholar.py | ok | - | 0 | 31.0s |
| Preprints (Europe PMC) | search_medrxiv.py | ok | - | 0 | 0.7s |
| YouTube Influencers | yt-transcript-scanner.py | ok | - | 0 | 3.6s |
| Bluesky | bluesky-search.py | ok | - | 0 | 0.5s |
| Org Scanner | org-scanner.py | ok | 6 | 12 | 13.7s |
| Voice Scanner | voice-scanner.py | ok | - | 0 | 1.3s |
| Newsletter Fetcher | newsletter-fetcher.py | ok | - | 1 | 9.5s |
| DSA Norway | ../aggregator/dsa_scraper.py | ok | - | 0 | 0.5s |
| SSM Sweden | ../aggregator/ssm_scraper.py | ok | - | 0 | 0.7s |
| ARPANSA Australia | ../aggregator/arpansa_scraper.py | error | - | 0 | 32.5s |
| STUK Finland | ../aggregator/stuk_scraper.py | ok | - | 0 | 1.6s |
| RIVM Netherlands | ../aggregator/rivm_scraper.py | ok | - | 0 | 1.2s |
| BAFU Switzerland | ../aggregator/bafu_scraper.py | ok | - | 0 | 1.6s |
| Israel MoEP | ../aggregator/israel_moep_scraper.py | ok | - | 0 | 0.5s |
| Site Patrol | site-patrol.py | ok | 19 | 0 | 18.2s |
Pipeline: discovery-master.py → candidate-processor.py → story-creator.py → Hugo build → deploy
| Tool | Script | Data Source | Output | Quick? |
|---|---|---|---|---|
| RSS Feeds | check-feeds.py | 16 verified RSS/Atom feeds | url-registry + eval-log | ✅ |
| PubMed | pubmed-search.py | NCBI E-utilities (8 queries) | url-registry | ✅ |
| EMF Portal | emf-portal.py | emf-portal.org news | url-registry | ✅ |
| Brave Search | brave-search.py | 4 batches (core, policy, emerging, cancer) | url-registry | — |
| Semantic Scholar | semantic-scholar.py | 8 EMF queries (free API) | url-registry | — |
| Preprints | search_medrxiv.py | Europe PMC (medRxiv/bioRxiv) | url-registry | — |
| YouTube | yt-transcript-scanner.py | 26 influencer channels | url-registry | — |
| Org Scanner | org-scanner.py | EHT, Phonegate, RRT, MWN | url-registry | — |
| Voice Scanner | voice-scanner.py | Moskowitz, Davis, Hardell | url-registry | — |
| Bluesky | bluesky-search.py | Bluesky social search | url-registry | — |
| Newsletters | newsletter-fetcher.py | Newsletter monitoring | url-registry | — |
| Site Patrol | site-patrol.py | Site availability checks | status only | — |
| Government Scrapers (no RSS available) | ||||
| DSA 🇳🇴 | dsa_scraper.py | Direktoratet for strålevern | url-registry | — |
| SSM 🇸🇪 | ssm_scraper.py | Strålsäkerhetsmyndigheten | url-registry | — |
| ARPANSA 🇦🇺 | arpansa_scraper.py | Australian radiation agency | url-registry | — |
| STUK 🇫🇮 | stuk_scraper.py | Finnish radiation authority | url-registry | — |
| RIVM 🇳🇱 | rivm_scraper.py | Netherlands RIVM | url-registry | — |
| BAFU 🇨🇭 | bafu_scraper.py | Swiss environment office | url-registry | — |
| MoEP 🇮🇱 | israel_moep_scraper.py | Israel environment ministry | url-registry | — |
| Candidate Processor | candidate-processor.py | Evaluates candidates → creates drafts |
| ↳ Quote Extraction | extract_quotes() | Two-tier: 1) finds text already in quotation marks (“...”, «...», „...“) with attribution detection — 2) keyword-matched sentences as fallback |
| Story Creator | story-creator.py | YAML → Hugo .md + SOME card |
| Publish Draft | publish-draft.py | drafts/ → stories/ + registry update |
| URL Registry | url-registry.py | Check/add/manage URLs |
| SOME Card | generate-card.js | Social media card PNG |
| Edit Stats | edit-stats.py | Git-based edit statistics |
| Frontpage Stats | frontpage-stats.py | Source type distribution |
| Verification Tools | ||
| Archive Source | archive-source.py | Downloads source article copy (txt + html) for verification |
| Verify Dates | verify-dates.py | Checks date field against source meta tags |
| Verify Quotes | verify-quotes.py | Fuzzy-matches quotes against archived source text |
| Verify Share Text | verify-share-text.py | Checks share_text length (X/Bluesky limits), Source citation, YouTube tag, via lines, EN/NO match |
| Verify Fields | verify-yaml-fields.py | Checks all required YAML fields are present and non-empty (source_language, title, date, icon, etc.) |
| Verify Structure | verify-structure.py | Checks study block completeness, share_text rules, quote attribution per source_type |
| Verify Dictionary | verify-dictionary.py | Checks title, summary, share_text, quotes for wrong-language terms (e.g. FHI in English text → NIPH) |
| Verify Topic | verify-topic.py | Checks that drafts matching a topic's auto_match rules have the correct topic category assigned |
| Verify No Body Text | verify-no-body-text.py | Checks every story/draft ends after YAML frontmatter; body text must be moved into structured fields |
| ● | https://www.digitaljournal.com/pr/news/indnewswire/electric-cars-electromagnetic | electromagnetic |
| ● | https://ehtrust.org/how-man-made-emfs-are-impacting-other-species-a-symposium-re | |
| ● | https://ehtrust.org/statement-by-the-environment-health-trust/ |
| ● | https://link.springer.com/article/10.1007/s10495-026-02260-y |
| ● | ICNIRP Workshop | icnirp |
| ● | Neu: Unser Ratgeber 2 „Mobilfunk, Digitalisierung, Alternati | |
| ● | Vortrag: Smart City Stuttgart – Fortschritt oder Risiko? |
| ● | (untitled) | candidate |
| ● | (untitled) | candidate |
| ● | (untitled) | candidate |
| ● | RF Animal Cancer Review Is #1 | included |
| ● | (untitled) | candidate |
| ● | (untitled) | excluded |
| ● | (untitled) | included |
| ● | (untitled) | included |
| ● | (untitled) | rejected |
| ● | “They Kept Telling Us What To Do” | included |
| ● | (untitled) | included |
| ● | NTP Lite: No Cancer Risk Seen in Japan | included |
| ● | (untitled) | excluded |
| ● | (untitled) | excluded |
| ● | (untitled) | candidate |
| ● | WHO Gets an ‘F’ on RF | excluded |
| ● | (untitled) | excluded |
| ● | (untitled) | excluded |
| ● | (untitled) | excluded |
| ● | Swimming Against the Current | excluded |
| ● | ICNIRP Workshop | included |
| ● | International Workshop on the Causes of Childhood Leukemia | candidate |
| ● | Serious game: Talk Science | excluded |
| ● | International seminar on Emerging technologies & Health | excluded |
| ● | BioEM - New calls for Short Term Mission | excluded |
| ● | Assessment of WHO-commissioned systematic reviews by BERENIS | excluded |
| ● | BioEM 2026 - Call for abstracts | excluded |
| ● | Course Bioelectromagnetic systems for innovative High Performance Sensing and Interaction strategies | excluded |
| ● | BioEM - Call for abstracts for Short Term Mission | excluded |
| ● | On our own behalf: restrictions on content | excluded |
| 📡 RSS Feeds | 16 verified feeds — Microwave News, EHT, SaferEMR, diagnose:funk, Flydal, EMF Portal, Phonegate, etc. | Daily |
| 🔬 PubMed | 8+ keyword queries across NCBI research database | 2×/week |
| 📚 EMF Portal (RSS) | News RSS + recent publications from emf-portal.org. See also: EMF Portal Bulk Data below. | 2×/week |
| 🔍 Brave Search | Core (15), bioeffects (12), Nordic (6) query batches | 2×/week |
| 📄 Semantic Scholar | 8 EMF queries, free API — academic papers from 2025+ | 2×/week |
| 📄 Preprints (Europe PMC) | medRxiv/bioRxiv preprints via Europe PMC API — RF, EMF, 5G health | 2×/week |
| 🎬 YouTube Transcripts | 7 channels, 15 keywords — needs API key for full 26-channel scanning | Per run |
| 🦋 Bluesky Search | 7 EMF queries via free public API — low activity so far | Per run |
| 🏢 Org Scanner | 12 EMF organization websites — RSS + HTML scraping (EHT, Phonegate, RRT, etc.) | Per run |
| 🧑🔬 Researcher Voices | 13 researchers — Moskowitz, Hardell, Davis, Pall, Havas, Leszczynski, etc. | Per run |
| 📰 Newsletter Fetcher | 8 sources — diagnose:funk, EMFSA, EHT, Microwave News, Phonegate | Per run |
| 🌐 Site Patrol | 19 websites — availability checks + new content detection | Per run |
| 📥 scrape.py | Full database export — 70 topic×frequency combos, 49,600+ articles as CSV+RIS. Repo: ~/projects/emf-portal-data/ | Manual |
| 🔄 emf-portal-diff.py | Compares scrape dates, finds new articles, cross-checks url-registry. Filters by topic, year. | Per scrape |
| 🔍 viewer.html | Local searchable table of all articles — filters, sort, highlighted marking. Private use only (EMF Portal copyright). | Local |
| 📈 /statistics/ | Public page with aggregate counts only — topic breakdown, year trends, languages, top journals. No article data reproduced. | Public |
| 🎯 discovery-master.py | Runs all 12 source tools in sequence. Registers new URLs in url-registry.json. Logs run stats to discovery-log.json (shown in Discovery Status above). |
| 📋 check-feeds.py | Evaluates each RSS item: emf_source / keyword_match / duplicate / no_match. Writes per-item verdicts to evaluation-log.json (shown in Evaluation Trail above). |
| 🏭 candidate-processor.py | Takes Brave Search batch candidates, fetches content, extracts quotes, generates YAML, creates Hugo drafts. Writes to evaluation-log.json. |
| 📝 story-creator.py | Converts YAML story specs into full Hugo markdown drafts (EN + NO). Called by candidate-processor or manually. |
| 🗄️ url-registry.py | Central URL dedup database (2566 URLs). Tracks status: candidate → included/rejected → published. |
| EMF/health subreddits — not monitored | |
| Twitter/X Search | API access needed |
| Government Scrapers | ARPANSA (AU), SSM (SE), STUK (FI), RIVM (NL), BAFU (CH), Israel MoEP — planned |
Coming later.
| Element | Existence validation | Existence valid. result | Content validation | Content valid. result |
|---|---|---|---|---|
| Summary | Yes | Present | No data | No summary validation data |
| Source | Yes | Elöverkänsligas Riksförbund | Field check | Source field only |
| Date | Yes | Date field present | No data | No validation data |
| Quotes | Yes | Quotes present in draft | No data | No validation data |
| Source type / category | Yes | NGO | No data | No validation data |
| Share text | Yes | share_text present | No data | No share-text validation data |
| Front page section YAML field: categories |
Yes | categories: policy |
Category mapping | Advocacy Derived from source_type/categories; for Mads/Larry review, not a factual source check. |
| YAML fields | No | No YAML completeness data | No data | No YAML field validation data |
| Element | Validated? | Validation result |
|---|---|---|
| Editorial (Larry) review | No data | Not reviewed |
| Dictionary / language | No data | Not checked |
| Topic / master card | No data | Not checked |
| No body text | No data | Not checked |
| Element | Existence validation | Existence valid. result | Content validation | Content valid. result |
|---|---|---|---|---|
| Summary | Yes | Present | Yes | pass — Larry summary check passed: concise, non-TODO, actor/source context present where requiredPrevious: pass — Larry summary check passed: concise, non-TODO, actor/source context present where required; pass — Larry summary check passed: concise, non-TODO, actor/source context present where required; pass — Larry summary check passed: concise, non-TODO, actor/source context present where required |
| Source | Yes | Orthop Rev (Pavia) | Field check | Source field only |
| Date | Yes | Date field present | Yes | confirmed — Matched: meta:citation_date, text:0Previous: confirmed — Matched: meta:citation_date, text:0; confirmed — Matched: meta:citation_date, text:0; confirmed — Matched: meta:citation_date, text:0 |
| Quotes | Yes | Quotes present in draft | Yes | confirmed — 3/3 verifiedPrevious: confirmed — 3/3 verified; confirmed — 3/3 verified; confirmed — 3/3 verified |
| Source type / category | Yes | Therapeutic | Yes | confirmed — source_type: TherapeuticPrevious: confirmed — source_type: Therapeutic; confirmed — source_type: Therapeutic; confirmed — source_type: Therapeutic |
| Share text | Yes | share_text present | Yes | pass — OK (167 chars)Previous: pass — OK (167 chars); pass — OK (167 chars); pass — OK (167 chars) |
| Front page section YAML field: categories |
Yes | categories: therapeutic |
Category mapping | Therapeutic EMF Applications Derived from source_type/categories; for Mads/Larry review, not a factual source check. |
| YAML fields | Yes | complete — All fields present | Yes | confirmed — All required fields presentPrevious: confirmed — All required fields present; confirmed — All required fields present; confirmed — All required fields present |
| Element | Validated? | Validation result |
|---|---|---|
| Editorial (Larry) review | Yes | pass — Title strong, summary concise, actors clear, no COI flagsPrevious: pass — Title strong, summary concise, actors clear, no COI flags; pass — Title strong, summary concise, actors clear, no COI flags; pass — Title strong, summary concise, actors clear, no COI flags |
| Dictionary / language | Yes | pass — No wrong-language terms foundPrevious: pass — No wrong-language terms found; pass — No wrong-language terms found; pass — No wrong-language terms found |
| Topic / master card | Yes | pass — No topic match (OK — not all stories belong to a topic) |
| No body text | Yes | confirmed — No body text |
| Element | Existence validation | Existence valid. result | Content validation | Content valid. result |
|---|---|---|---|---|
| Summary | Yes | Present | Yes | pass — Larry summary check passed: concise, non-TODO, actor/source context present where requiredPrevious: pass — Larry summary check passed: concise, non-TODO, actor/source context present where required; pass — Larry summary check passed: concise, non-TODO, actor/source context present where required; pass — Larry summary check passed: concise, non-TODO, actor/source context present where required |
| Source | Yes | Mikhaila Peterson (YouTube) | Field check | Source field only |
| Date | Yes | Date field present | Unverifiable | no_source_dates — No dates in sourcePrevious: no_source_dates — No dates in source; no_source_dates — No dates in source; no_source_dates — No dates in source |
| Quotes | Yes | Quotes present in draft | No | issues — 0/2 verifiedPrevious: issues — 0/2 verified; issues — 0/2 verified; issues — 0/2 verified |
| Source type / category | Yes | PeoplesVoice | Yes | confirmed — source_type: PeoplesVoicePrevious: confirmed — source_type: PeoplesVoice; confirmed — source_type: PeoplesVoice; confirmed — source_type: PeoplesVoice |
| Share text | Yes | share_text present | Yes | pass — OK (138 chars)Previous: pass — OK (138 chars); pass — OK (138 chars); pass — OK (138 chars) |
| Front page section YAML field: categories |
No | categories: — |
Category mapping | People's Voice Derived from source_type/categories; for Mads/Larry review, not a factual source check. |
| YAML fields | Yes | complete — All fields present | No | issues — missing categories — List of categories; missing country_code — ISO country codePrevious: issues — missing categories — List of categories; missing country_code — ISO country code; issues — missing categories — List of categories; missing country_code — ISO country code; issues — missing categories — List of categories; missing country_code — ISO country code |
| Element | Validated? | Validation result |
|---|---|---|
| Editorial (Larry) review | No | issues — source_type=PeoplesVoice but no editor_note — verify no COIPrevious: issues — source_type=PeoplesVoice but no editor_note — verify no COI; issues — source_type=PeoplesVoice but no editor_note — verify no COI; issues — source_type=PeoplesVoice but no editor_note — verify no COI |
| Dictionary / language | Yes | pass — No wrong-language terms foundPrevious: pass — No wrong-language terms found; pass — No wrong-language terms found; pass — No wrong-language terms found |
| Topic / master card | Yes | pass — No topic match (OK — not all stories belong to a topic) |
| No body text | Yes | confirmed — No body text |
This previews the videos still marked as draft in videos.yaml. It uses the same carousel format as the front page, so you can inspect how they will look live.
| Tool | Technology | What it scans | Status |
|---|---|---|---|
| check-feeds.py | urllib + XML | 13 RSS/Atom feeds | ✅ Daily |
| pubmed-search.py | NCBI E-utilities | PubMed (last 7 days) | ✅ Daily |
| emf-portal.py | urllib + XML/HTML | emf-portal.org news | ✅ Daily |
| brave-search.py | Brave Search API | Web search batches | 🔄 Manual |
| org-scanner.py | urllib + HTML scraping | 14 org websites | ❌ Disabled |
| voice-scanner.py | urllib + RSS | 13 researchers | ❌ Disabled (mostly search suggestions) |
| yt-transcript-scanner.py | youtube-transcript-api | YouTube transcripts | ❌ Disabled |
| bluesky-search.py | AT Protocol | Bluesky posts | ❌ Disabled |
| site-patrol.py | requests | Our site uptime | ❌ Disabled |
| Feed | Source | Status |
|---|---|---|
| Microwave News | microwavenews.com | ✅ |
| EHT (Devra Davis) | ehtrust.org | ✅ |
| Phonegate (Marc Arazi) | phonegatealert.org | ✅ |
| EMF Portal (News + Pubs) | emf-portal.org | ✅ |
| SafeRemR (Moskowitz) ×2 | saferemr.com | ⚠️ Atom parser broken |
| diagnose:funk | diagnose-funk.org | ✅ |
| RRT | radiationresearch.org | ✅ |
| Einar Flydal | einarflydal.com | ✅ |
| FHI | fhi.no | ✅ |
| UKHSA | gov.uk | ✅ |
| BfS (Germany) | — | ❌ No feed found |
| ANSES (France) | — | ❌ No feed found |
| Tier | Voice | Actual scanning |
|---|---|---|
| ⭐ 1 | Joel Moskowitz (SafeRemR) | ⚠️ RSS feed — Atom parser broken |
| ⭐ 1 | Devra Davis (EHT) | ✅ RSS via check-feeds + org-scanner |
| ⭐ 1 | Lennart Hardell | ❌ Search suggestion only |
| 📝 2 | Ron Melnick | ❌ Search suggestion only |
| 📝 2 | Martin Pall | ❌ Search suggestion only |
| 📝 2 | Henry Lai | ❌ Search suggestion only |
| 📝 2 | Dominique Belpomme | ❌ Search suggestion only |
| 📝 2 | Einar Flydal | ✅ RSS via check-feeds |
| 📝 2 | Suzanne Burdick (CHD) | ✅ Via org-scanner (disabled) |
| 📝 2 | Marc Arazi (Phonegate) | ✅ RSS via check-feeds |
| 📚 3 | Meike Mevissen | ❌ Search suggestion only |
| 📚 3 | Anthony B. Miller | ❌ Search suggestion only |
| 📚 3 | David Carpenter | ❌ Search suggestion only |