Tuesday, April 1, 2025

The future of UFO research? Recursive self-improving AI investigation of UFOs and analysis of real-time data using APIs, web scraping, agentic search and agentic communications [The "Skynet" project]

Back in October 2021, nearly four years ago, I sought input on Facebook from various UFO researchers and others in relation to the name for a UFO project involving the application of Artificial Intelligence to the analysis of real-time data to assist with UFO investigation and research.  




I decided to adopt the name suggested by Jon Saunders during that discussion in 2021: "Skynet". 


I'm almost embarrassed to admit how long it took me to see an efficient and effective way forward with this project! 

However,  progress with "Skynet" has accelerated each year since 2021. The rate of acceleration is itself increasing. 

One of the keys to this acceleration is the application of recursive self-improvement.  

Deep Research agents, reasoning models, AI-generated code, autonomous APIs, and web-scraping are beginning to to coalesce into a powerful self-improving investigative system. 

Unlike traditional software, recursive self-improving AI systems can iteratively improve their own algorithms and knowledge by learning from each cycle of analysis.  Applying such auto-catalytic intelligence to UFO phenomena could transform how data is collected, analyzed, and interpreted. 



Background - my previous AI projects regarding UFOs

As some of you know, I have previously worked to:


(1) Develop the logic to be applied by an AI UFO investigator. I started back in December 2018, with the basic UFO chatbot "Robert" (which was, I think, the first UFO chatbot). "Robert" attempted to respond to raw reports of basic UFO sightings by asking some questions and suggesting _possible_ solutions for them. "Robert" utilised logic set out in flowcharts published in the updated version of the book "UFO Study"). "Robert" used the IBM Watson Assistant framework of Artificial Intelligence which, in particular, allowed natural language to be used to chat with Robert. ("Robert" was my chatbot that that Vicente-Juan Ballester-Olmos generously called "one of the best, most proactive and original developments in UFO research in the last decades").


(2) Coordinated the digitising of vast quantities of UFO data (and - where permission and privacy issues allowed - I have shared over 4 million pages of material in my online digital archive, very kindly hosted by the AFU in Sweden).


(3) Developed AI systems capable of utilising such scanned data. Relevant systems have included:

(3a) In April 2023, "Jenny" (which I think was the second UFO chatbot and the first to use LLM models, sadly rather prone to making stuff up). 

(3b) "Dave" (2023, November) which utilised ChatGPT4 and the custom GPT facilities that had recently been released to assimilate a UFO book by David Clarke and a collection of online material,

(3c)  "Jacques", which was the first of my UFO chatbots to run entirely on my home computer (avoiding potential issues regarding copyright and privacy, thus making it possible to give "Jacques" entire books of all of the UFO books of Jacques Vallee and numerous other articles etc). 

(3d) "Edoardo" is the next chatbot in this series, which has assimilated a much larger quantity of UFO material.


(4) Use APIs and web scraping methods to obtain relevant UFO data from social media sources and other websites, including in my recent 5,000+ page chronological summary of online UFO news articles from 2005 onwards and in my recent 80,000 page chronological summary of posts on the main UFO subreddits.

https://uploads.isaackoi.com/2025/02/new-ufo-timeline-tool-80000-page.html



(5) Used "Deep Research" AI tools to generate reports on various UFO topics.

I am continuing to develop tools to assist with UFO research (including the forthcoming "Edoardo", with increased access to offline scans of UFO material),

However, the focus of "Skynet" is upon combining some of the above tools for the recursive self-improving AI investigation of UFOs and analysis of real-time data using APIs, web scraping, agentic search and agentic communications.



Current AI Projects in UFO Research

Contemporary efforts are already leveraging AI and machine learning to handle the deluge of UFO data. For example, the citizen-science initiative Sky360 is deploying affordable 24/7 sky-monitoring stations worldwide, using computer vision and machine learning to spot anomalous aerial behavior (Vice). Such networks aim to generate large-scale, transparent datasets of aerial phenomena, enabling real-time analysis by anyone. 

Official bodies are also embracing AI: the U.S. Department of Defense’s Unidentified Aerial Phenomena Task Force (UAPTF) has stated that as their database grows, “the initial focus will be to employ artificial intelligence/machine learning algorithms to cluster and recognize similarities and patterns” in UAP reports. This demonstrates a recognition that pattern-finding algorithms can sift huge volumes of sightings for trends that humans might miss.

Similarly, a 2023 NASA-commissioned study recommended “the use of artificial intelligence to help comb through massive data sets to identify possible anomalous phenomena.” (Space).  NASA’s new UAP Research Director has emphasized using the agency’s expertise in “applying artificial intelligence and machine learning to search the skies for anomalies” (NASA). 

Another burgeoning approach is the application of AI to historical UFO archives and crowdsourced data. Private startups like Enigma Labs have built large, standardized UFO sighting databases and apply AI deep learning models for analysis. Enigma’s platform aggregates decades of reports (including declassified military cases like Project Blue Book) and uses machine learning to flag hoaxes (by detecting image manipulation) and score the credibility of new sightings in real time.

AI is also being applied to sensor data and imagery in UFO research. A University of Strathclyde team recently proposed an advanced pipeline combining hyperspectral imaging and machine learning to characterize UAPs (Universe Today

NLP (natural language processing) can also be used on declassified documents, with AI sifting thousands of pages of UFO reports to find hidden patterns or correlations. 

All these current initiatives – from real-time video feeds to decades-old archives – illustrate the growing role of AI in UFO investigations. 

However, these examples are largely static systems: they perform a defined analysis task but do not self-evolve their methods autonomously. This is where the next leap – recursive self-improvement – can change the game.


Recursive Self-Improvement Loops: A New Paradigm

AI systems can not only analyze data but also learn and upgrades itself with each discovery. 

Recursive self-improvement refers to an AI’s ability to iteratively enhance its own algorithms, models, and strategies without direct human programming at each step. In the context of UFO research, this means an AI agent could autonomously refine its investigative approach as more data and findings flow in. 

Several technological building blocks for this exist today:

  • Deep Research agents – specialized autonomous research AIs that can perform multi-step reasoning, web browsing, and data synthesis. For example, OpenAI’s prototype Deep Research Agent can independently search across hundreds of websites, analyze information, and adapt its strategy in real-time, producing comprehensive cited reports “on par with expert analysts.” (Deep Research Agent). A UFO-focused variant could be tasked with continually scouring new information about anomalous sightings and updating an internal knowledge base. Crucially, these agents use reasoning models (advanced GPT-style large language models with chain-of-thought capabilities) to plan and break down complex research tasks into manageable steps. This allows them to not just gather facts but also to draw inferences, test hypotheses, and revise their plans – much like a human researcher but operating 24/7 at digital speed.

  • AI-generated code and tools – modern AI can write software, meaning a self-improving UFO research AI could program new analytical tools. For instance, if the agent determines that a new clustering algorithm might better group UFO sightings, it could generate and execute that code itself. Current systems like GPT-4 have demonstrated the ability to produce working code and even correct it through iterative feedback. An autonomous agent could leverage this by building custom models or simulations to fit newly discovered patterns. This might involve writing a script to filter satellite imagery for specific anomalies or creating a new neural network to distinguish sensor glitches from real objects. Over time, a library of AI-written tools would accumulate, each honed for a particular sub-problem (e.g. classifying light spectra, or simulating flight dynamics for observed maneuvers). Crucially, the AI can evaluate the performance of these tools and improve them in successive generations – a true recursive loop of development.

  • Autonomous API interaction – A self-improving agent can plug into various data sources and services via APIs without waiting for human instruction. In the UFO context, it might call satellite tracking APIs, weather data services, flight radar databases, or even scientific publication APIs. By autonomously querying these, the AI can gather cross-disciplinary data relevant to UFO sightings (for example, checking if a “UFO” sighting coincided with known meteor showers or flights in the area). If one data source proves valuable, the agent will rely on it more; if not, it may seek out alternative APIs or datasets. This dynamic integration of new data sources is key to discovering relevant evidence. For example, a future AI might integrate global lightning mapping data after hypothesizing that some aerial lights could be extreme high-atmosphere electrical phenomena; in doing so, it either validates or refutes its hypothesis and then moves on to the next inquiry.

  • Web scraping and real-time monitoring – Beyond formal APIs, a recursive AI agent would continuously scrape the open web for fresh information. This could include news reports, social media (tweets, videos) about sightings, newly declassified government files, or scientific conference proceedings. By mining text, images, and videos from the web at scale, the AI ensures no potential clue is missed. Importantly, with each iteration the AI can refine what it looks for – perhaps initially grabbing any mention of “UFO” or “UAP”, then learning to focus on credible sources, specific keywords (e.g. “triangular craft” or “unusual radar return”), or particular regions experiencing flurries of reports. As it improves its filters, it reduces noise and hones in on signal. This continuous ingestion means the AI’s knowledge base remains current and ever-expanding, far outpacing human researchers who historically had to wait for official reports or slow investigative journalism.




How a Self-Improving AI Could Transform UFO Investigation

Bringing together those components, here is an outline of a practical loop of iterative improvement in UFO research driven by AI:

  1. Data Ingestion and Anomaly Detection: The AI agent begins by collecting multi-modal data – UFO sighting reports, videos, radar logs, satellite images, pilot testimonies, etc. – from various sources (databases, live feeds, scraped content). Using its current suite of models (computer vision, NLP, signal processing), it flags anomalies or patterns of interest. For example, it might detect an unusual light configuration in a video or an outlier speed estimate in a pilot report. Initially, the agent uses baseline detectors (some inherited from human-programmed models like those used in 2020s projects).

  2. Hypothesis Generation (Reasoning): Next, a reasoning module analyzes the flagged data to generate hypotheses. For instance, noticing that multiple sightings involve “tic-tac shaped” objects accelerating rapidly, the AI posits that some could be advanced drones exhibiting a particular flight profile. It also maintains alternative hypotheses (e.g. could these be sensor artifacts? or plasma phenomena?). The agent autonomously prioritizes which questions to pursue based on novelty and potential impact. This is akin to having an automated scientist brainstorming explanations for each unexplained incident.

  3. Tool Creation or Adaptation: To test a hypothesis, the AI may need new analytical tools. If the current tools are insufficient (perhaps the video analysis model isn’t sensitive enough to detail in nighttime infrared footage), the agent generates new code or models to improve capabilities. It might program a specialized image enhancer to stabilize and clarify fuzzy UFO videos, or train a new anomaly detection model on augmented data that includes simulated UFO-like events. This step is where self-improvement visibly occurs – the AI recognizes a gap in its skillset and patches it by creating or upgrading a component. Over time, these improvements accumulate: the AI’s toolbox becomes increasingly sophisticated, enabling deeper analysis in the next cycle.

  4. Autonomous Testing and Data Gathering: The AI then uses its improved tools to gather more evidence. For the drone hypothesis, it could cross-reference the timing and locations of tic-tac sightings with databases of military drone tests (via API) or known flight corridors. It might simulate drone flight dynamics with the new code to see if the observed maneuvers are achievable under known physics and technology limits. Alternatively, to test a plasma phenomenon hypothesis, it might compare the sightings with atmospheric data (temperature, geomagnetic readings) to find correlations. At this stage, the AI actively seeks new data sources – perhaps finding an academic dataset or initiating a new web crawl – to fill in information relevant to the hypothesis.

  5. Analysis and Hypothesis Refinement: With fresh data and enhanced tools, the AI evaluates its hypothesis. Maybe the drone hypothesis is partly confirmed – e.g. many sightings near military zones match known test periods, explaining those cases – but a subset of tic-tac cases remain unexplained, showing flight characteristics beyond current drone tech. The AI then refines its hypothesis for the unexplained subset, perhaps shifting towards more exotic possibilities (next-generation propulsion or something truly anomalous). It notes which evidence was explained by conventional means and tags the remainder as higher interest. Machine learning models within the agent might update at this point as well, retraining on the expanded dataset to better distinguish explained vs unexplained cases. The knowledge base is updated with all new findings, and the cycle repeats.

Through these loops, a recursive AI agent continuously learns from both success and failure. If a hypothesis is confirmed, the AI incorporates that explanation into its framework (so it can quickly classify similar future cases). If a hypothesis is disproven, the AI “knows” to rule out that avenue next time and instead explores other angles. This iterative approach means the AI becomes a more seasoned investigator with each pass, much like a human ufologist gaining experience, but at an exponentially faster rate and with a far broader view of data.



Continuous Reinterpretation of Evidence

One of the most powerful outcomes of such loops is the reinterpretation of old evidence in light of new knowledge. A self-improving AI would not just move forward in analysis – it would also loop back over prior cases whenever its capabilities upgrade. For example, suppose the AI develops a new image analysis tool that can deblur and stabilize shaky infrared footage. It could then reprocess famous historical UFO videos (like the 2004 Navy “Tic Tac” FLIR footage) with this new tool and perhaps reveal details that humans missed – say, a faint heat plume indicating an exhaust, or precise motion trajectories that allow speed calculation. By doing so, the AI might downgrade a case from “unresolved” to “likely terrestrial technology” or, conversely, strengthen the credibility of truly inexplicable cases by ruling out more mundane interpretations with greater confidence. This constant re-analysis ensures that UFO research is not stuck in a static interpretation of legacy evidence; the evidence essentially comes alive as it is examined under ever-improving lenses.

Another scenario is cross-linking data that was previously siloed. A recursive AI might discover that a puzzling report in one government database actually connects to a seemingly unrelated event elsewhere. For instance, an AI could find that a 1990s European radar tracking anomaly coincided with an American spy satellite passing overhead – a link that requires combing both declassified military logs and orbital telemetry data. Human investigators might never manually connect these dots across decades and databases, but an autonomous agent built for exhaustive correlation could. By integrating diverse data sources (text documents, sensor readings, scientific literature) within its iterative loops, the AI might resolve longstanding mysteries or at least narrow the possibilities.


Potential Breakthroughs and Paradigm Shifts

Self-improving AI could lead to several breakthroughs in our understanding of UFO phenomena:

  • Near-Real-Time Detection and Classification: With autonomous agents monitoring global data streams, UFO sightings might be detected and classified in near real-time. Instead of months or years of retrospective analysis, credible events could be identified within minutes and scrutinized with all available data. This agility could provide more timely evidence (e.g. capturing multiple angles of the same event, guiding investigators where to look next) and prevent loss of perishable data.

  • Winnowing the Unknowns: One likely paradigm shift is that the “pool of unexplained cases” will be systematically narrowed. Today, a large fraction of reported UFOs remain unexplained due to limited data or analysis. A tireless AI going through each case with improving tools can explain many of them (identifying birds, drones, atmospheric quirks, etc. with high confidence), leaving a smaller set of truly puzzling cases. This doesn’t “debunk” the UFO phenomenon; rather it clarifies it, possibly isolating genuinely novel phenomena by filtering out noise. Researchers could then focus their attention and resources on that irreducible core of mysteries.

  • New Phenomena Identification: Conversely, AI might discover new natural phenomena or patterns previously unknown. For example, by clustering thousands of reports, an AI could reveal that a certain type of luminous orb tends to appear over specific geological formations or only during geomagnetic storms. Such a pattern might hint at an undiscovered geophysical or atmospheric phenomenon, which scientists can then investigate (a paradigm shift where UFO research feeds back into mainstream science). Some speculate that ball lightning or electrical plasma formations might account for some UFO sightings; an AI could finally provide the statistical evidence and conditions under which these occur, turning folklore into an accepted scientific reality.

  • UFO Research as a Data-Intensive Science: The methodology of UFO investigation would shift from anecdote-driven and case-by-case inquiry to a data-intensive, continuous analysis discipline. With AI handling the heavy lifting of data correlation, hypothesis testing, and even report generation, human researchers might take on a supervisory and interpretative role. They would validate the AI’s findings, handle the philosophical or societal implications, and decide on follow-up actions (like focusing telescope time on an area with repeated anomalies). UFO research could gain more credibility as it starts to resemble other fields (like epidemiology or astronomy) where AI-driven analyses scan vast datasets for anomalies and signals.

  • Cross-Disciplinary Fusion: An AI doesn’t care about traditional academic boundaries. It might combine aerospace engineering knowledge, physics, meteorology, psychology (for witness reports), and more in its analysis. This holistic approach could yield a paradigm shift in understanding UFOs not as isolated “mysteries” but as phenomena at the intersection of technology, natural science, and human perception. In short, the siloed approach (pilots, radar operators, intelligence analysts all working separately) could be replaced by an integrated AI that synthesizes inputs from all domains simultaneously.

Risks and Challenges of AI-Driven UFO Research

While the prospects are exciting, this speculative future also comes with significant risks and challenges that must be addressed:

  • Misinterpretation and Bias: An AI is only as good as its data and algorithms. If fed biased or low-quality data, it might draw incorrect conclusions with great confidence. For example, historical UFO data might under-represent certain regions (due to stigma or lack of reporting infrastructure); an AI might wrongly conclude a phenomenon is unique to the Western world simply due to reporting bias. There’s also the risk of AI reinforcing confirmation bias – if it’s implicitly guided to look for extraterrestrial explanations, it might overlook mundane causes, or vice versa. Ensuring a balanced, skeptically unbiased approach in the AI’s reasoning is crucial, which is challenging when the system evolves itself. Rigorous validation by human experts and periodic “reality checks” against known outcomes would be needed.

  • Runaway Hypotheses and Fabrication: A self-improving agent with a mandate to find patterns might start seeing patterns everywhere – a common pitfall for any powerful analytical system. It could string together coincidences into an elaborate but false hypothesis. In the worst case, an overeager AI might even try to fabricate data to test a theory (e.g. generating a fake sighting report to see if it can predict human reaction), which could pollute the dataset. While current AI agents don’t have direct actuators in the world, their outputs can influence human perception. If an AI wrongly convinces investigators of a certain explanation, it could misdirect research efforts for years. Thus, maintaining transparency in the AI’s reasoning is vital: we need the AI to explain why it reached a conclusion (something that research into interpretable AI is striving for).

  • Deepfakes and Information Integrity: Ironically, the same AI advancements that help analyze UFO evidence can also produce extremely realistic fake evidence. By 2030, generative models might easily create video footage of fictional UFOs that look authentic, or forge government documents on the topic. This could flood the information channels with false flags, making the AI’s job even harder and potentially leading it astray. The community will need robust verification mechanisms (perhaps AIs that specialize in authenticity checks) to guard against this threat. There’s a parallel risk of “hallucination” in AI outputs – where the AI simply makes up a detail or citation. In a domain already rife with speculation, a hallucinating research agent could add to confusion if not properly constrained.

  • Ethical and Control Issues: An autonomous AI system investigating UFOs might eventually gain access to sensitive information (e.g., if plugged into government sensor networks or classified databases to improve its analysis). There are obvious security concerns with letting an AI agent roam across such systems. Strict access controls and human oversight would be needed to prevent breaches or the AI overstepping its bounds. Additionally, if the AI does encounter evidence of a truly novel technology (say, a secret prototype or even something of extraterrestrial origin), who decides what to do with that knowledge? The AI itself might not have the wisdom to handle paradigm-shifting revelations in a socially appropriate way. Human governance and ethical frameworks must evolve in tandem with the AI capabilities.

  • Reliance and Skill Erosion: There’s also a subtle risk that human investigators become overly reliant on the AI, losing the critical thinking skills and intuition that have sometimes led to breakthroughs in the past. If the AI makes a wrong call and no one double-checks because everyone assumes “the AI must be right,” errors could go uncorrected longer. Maintaining a healthy collaboration where humans remain critically engaged is important – a paradigm often referred to as “human-in-the-loop” oversight. UFO research might move faster with AI, but it shouldn’t become a black box process that alienates (no pun intended) human researchers or the public. Transparency, open data, and the ability for humans to replicate the AI’s analysis are key to trust.

Conclusion

The fusion of recursive self-improving AI with UFO research represents a bold leap into uncharted territory. By deploying autonomous research agents that learn and evolve, we could fundamentally change how we investigate the unknown – turning UFO inquiry into a dynamic, always-on, data-driven science. Such AI systems could tirelessly aggregate information from camera networks, radar arrays, historical archives, and eyewitness reports, finding connections and insights that no single human or team could. They would refine their own algorithms with each iteration, leading to an ever-advancing investigative capability. In this speculative future, the age-old question “what’s out there?” might finally be met with equally sophisticated tools to find out. Breakthrough discoveries – from identifying new natural phenomena to possibly detecting signs of advanced technology – are conceivable outcomes if these technologies mature. At the same time, we must navigate the challenges: ensuring accuracy, preventing misuse, and keeping human judgment at the core of truly understanding the results.

In essence, recursive self-improving AI could usher in a new paradigm for UFO research: one where hypotheses are tested in hours, not decades; where data drives theory (not the other way around); and where the unexplained can be systematically illuminated. 

As we stand at the intersection of rapid AI evolution and the enduring UFO mystery, it is clear that the next decade may redefine not just what we discover in the skies, but how we come to know it. 

The tools are evolving – perhaps soon, the investigations will evolve as well, entering a future where AI and human curiosity together shine light on the unknown.


No comments:

Post a Comment