[Event Report] Airion Co-Hosted "Industrial AI Night" with ONESTRUCTION — the Front Lines of AI Development in Construction and Manufacturing

On June 30, 2026, ONESTRUCTION Inc. and Airion Inc. jointly held an event for engineers, "Industrial AI Night: AI Development Today — the Front Lines of AI Development in Construction and Manufacturing," at ONESTRUCTION's Tokyo office.

Engineers from both companies, each developing domain-specific AI in different fields — construction and manufacturing — took the stage. They shared not only the technical ingenuity behind their model development, but also their approaches to data creation, evaluation design, building training environments, and the challenges of moving toward real-world operation.

We would like to once again thank everyone who attended, as well as Weights & Biases for sponsoring the event as our food sponsor.

[Event Overview]

"Industrial AI Night: AI Development Today — the Front Lines of AI Development in Construction and Manufacturing"
Date and time: June 30, 2026, 19:00–20:00
Venue: ONESTRUCTION Inc. Tokyo Office
Co-hosts: ONESTRUCTION Inc. and Airion Inc.
Sponsor: Weights & Biases

What Is "Industrial AI Night"?

Airion and ONESTRUCTION have both been selected for Cycle 3 of GENIAC, the domestic foundation model development support program promoted by Japan's Ministry of Economy, Trade and Industry and NEDO.

Airion has been developing LLMs specialized for manufacturing, and ONESTRUCTION for construction — so-called domain-specific models tailored to particular areas of expertise.

Although their target industries and workflows differ, the two companies share a common challenge.

In specialized domains where sufficient training data does not exist on the internet and existing large language models contain almost no relevant knowledge, how do you create training data, evaluate models, and raise accuracy to a level that can actually be used in the field?

This event was planned to share the insights, techniques, and lessons learned from failures gained through this development process — without watering down the technical detail.

It was held as a small, capacity-limited gathering, structured not just around one-way talks but around open exchange between speakers and participants on concrete development challenges.

What Airion and ONESTRUCTION Have Been Working On

In the first half of the event, both companies presented overviews of the domain-specific LLM development they have pursued under GENIAC Cycle 3.

Airion: A Manufacturing-Specific LLM That Automatically Generates Ladder Programs

Airion is developing an LLM that automatically generates ladder programs — the dedicated language for PLCs (programmable logic controllers), which control production equipment in factories.

Using Qwen3 as the base model, Airion has built a distributed training infrastructure using Megatron-LM, verl, and FSDP2.

For model training, we adopted the following three-stage pipeline.

Continued pre-training (CPT)
Supervised fine-tuning (SFT)
Reinforcement learning (RL)

Ladder programs are a specialized domain with extremely little publicly available information on the web, and they are barely represented in the training data of existing large language models.

As a result, simply collecting public data — as one might for a general-purpose programming language — is not enough to build an adequate training environment.

Airion built its training data, evaluation datasets, and evaluation environment based on the expertise and real-machine data of the production equipment manufacturers it partners with.

In actual field deployments, we have confirmed that the system reduces program creation time by roughly 20%.

ONESTRUCTION: A Construction-Specific LLM That Generates IDS from BIM

ONESTRUCTION presented an LLM that automatically generates IDS (Information Delivery Specification) — checklists for verifying attributes — from BIM (Building Information Modeling).

IDS is a relatively new technology, with version 1.0 appearing around 2024, and here too there is almost no reference data available on the internet.

ONESTRUCTION injects industry knowledge from inside and outside the company into the model through continued pre-training, and then performs SFT and RLVR.

The team also held multiple workshops with in-house domain experts, working through everything from defining the tasks to be solved to creating the evaluation datasets.

For the task of verifying the attribute information needed for structural and energy calculations in BIM, this work has led to a 55.53% reduction in verification time.

Despite operating in different industries, the session made clear that both companies have confronted the same wall — how to create specialized data that barely exists in the world — each with their own distinct approach.

Panel Discussion: "The Messy Reality of AI Development"

In the second half of the event, four AI engineers from the two companies held a panel discussion.

[Panelists]

Koyo Hidaka
Manager, AI Strategy Unit, ONESTRUCTION Inc.

Ryo Kanazawa
AI Strategy Unit, ONESTRUCTION Inc.

Takahide Yoshihira
AI Engineer, Airion Inc.

Kotaro Nomura
AI Engineer, Airion Inc.

The panel candidly shared not only technical achievements, but also the problems they actually ran into during development and the lessons learned from failures.

The Hardest Parts of Development

Airion explained that building the training environment itself was one of the biggest hurdles.

Even after adopting a distributed training framework like Megatron-LM, simply setting up the environment does not guarantee the expected training performance.

Accuracy and training efficiency cannot be judged until you actually train models and compare runs under different conditions.

When integrating proprietary training methods into existing libraries, the team also ran into dependency and compatibility issues between libraries.

Furthermore, ladder programs are a domain in which existing models have almost no knowledge.

Unlike other domains, it was difficult to rely on large-scale synthetic data generation by existing models, so the training corpus had to be built from scratch.

ONESTRUCTION noted that continuous consultation with domain experts was indispensable throughout the entire development process.

In-house experts had to be involved from the task-definition stage — deciding what the model should be made to solve — and the data for the evaluation benchmark was created by hand over several days.

Both companies' experiences pointed to the same conclusion: in specialized-domain AI development, success depends not just on models and GPUs, but on collaboration with people who hold field knowledge.

How the Evaluation Benchmarks Were Designed

In AI development for specialized domains, existing general-purpose benchmarks cannot necessarily be used as-is.

You therefore have to start by defining the evaluation task itself — what counts as correct, and what should be measured.

ONESTRUCTION explained that even after narrowing down the target task, a large number of patterns exist across scenarios and modalities.

Through workshops with domain experts, they organized real field data and designed their evaluation set with a clear standard in mind: a perfect score should mean the model is usable in the field.

At the same time, the model's behavior cannot be guaranteed for anything outside the defined benchmark.

They also candidly reflected on discovering gaps in task coverage mid-development and hurriedly adding benchmarks.

Airion described how evaluating the task datasets built from real-machine data was itself a major challenge.

Using a general-purpose large language model as the evaluator, without modification, may not produce appropriate results because it lacks domain knowledge of PLCs and ladder programs.

Airion therefore combined a rubric that organizes the evaluation criteria with a mechanism that dynamically references manufacturers' manuals.

A question came from the audience: "Who verifies the validity of the rubric itself?"

In response, we explained that rather than having experts score every output in detail, we ask them to do a lightweight check of whether the overall scores feel roughly right — balancing evaluation quality against the burden on experts.

In domain-specific AI, the evaluation system itself becomes a development target, not just the model.

The discussion between the two companies brought that importance into sharp relief once again.

Unglamorous but Important Tricks That Improved Accuracy

In this session, each panelist shared concrete techniques that led to accuracy improvements.

Airion shared its approach of routing data to different training stages depending on data quality.

High-quality data is used for post-training, while data containing a certain amount of noise is used on the pre-training side, making effective use of limited data.

We also explained that by carefully tuning rewards and penalties in reinforcement learning, we improved both compile rate and pass@1 by several points compared with SFT alone.

On the other hand, if the reward design is flawed, the model can end up optimizing simply to avoid compile errors rather than to write correct programs.

This is a difficulty peculiar to reinforcement learning, and it requires continuously checking what the model is actually learning.

ONESTRUCTION explained that they made the quality and correctness of the evaluation data their top priority, repeatedly verifying it with a "question everything" mindset.

They also shared an experiment in which domain-independent reasoning generated by a general-purpose open model was combined with ground-truth SFT outputs to create pseudo-reasoning data, which was then used for RLVR.

The result was reasoning that read as if the model genuinely understood the contents of the IDS.

Ingenuity in Training Data Creation — and Honest Stories of Failure

The session that generated the most buzz in the room was the sharing of failure stories from development.

Airion described a case where evaluation was run while the compiler used for syntax validation contained a bug that, under certain conditions, always returned "success."

Abnormally high accuracy in the evaluation results tipped the team off, leading to the discovery of the defect on the compiler side.

When a model's accuracy is higher than expected, before celebrating it as an achievement, you have to suspect problems in the data, the evaluation environment, or the evaluation code.

This experience reaffirmed the importance of continuously validating not just the model but the entire evaluation pipeline.

ONESTRUCTION shared a failure of their own: scoring was performed without excluding the reasoning tokens at evaluation time, which kept producing scores lower than the model's actual performance.

After excluding the reasoning portion and re-evaluating, they were able to confirm the correct scores.

It is not unusual for what looks like an accuracy problem to actually be a problem with the evaluation implementation rather than the model.

Both companies' failure stories show that in domain-specific AI development, training, inference, and evaluation must be validated as a single integrated system.

The Trade-off Between Model Size, Inference Cost, and Accuracy

The final theme was a discussion of model size and inference environments with real-world operation in mind.

In the PLC control domain that Airion targets, generated programs can directly drive physical equipment.

This makes it hard to justify sacrificing significant accuracy for faster inference; among speed, cost, and accuracy, safety and correctness must take priority.

Currently, assuming usage by around 10–20 users, we are considering a configuration that can run inference on a single GPU in the 80 GB VRAM class.

In cloud environments, inference libraries such as vLLM and SGLang can keep the slowdown within acceptable bounds even as model size grows.

In contrast, when deploying to local environments inside factories or on-premises environments, the available compute resources and infrastructure are constrained.

Having to design not just for model performance but also for the customer's equipment, user count, security requirements, and operating costs is a challenge unique to AI for manufacturing.

ONESTRUCTION shared its current view that rather than simply continuing to host large models — including the model developed in GENIAC Cycle 3 — the business model and user experience need to be further refined.

To make domain-specific models stick in real businesses and workflows, you have to design not only for model performance, but for when and how users derive value.

Sponsorship and Product Introduction by Weights & Biases

For this event, Weights & Biases sponsored the venue setup and provided food for the networking session.

After the panel discussion, Mr. Kamata of Weights & Biases took the stage and introduced the company's latest product updates, including a freely available white paper on reinforcement learning, a feature for tracking the activity history of coding agents, and the agent feature "Aria."

At the networking session that followed the technical sessions, speakers and participants exchanged views actively over light refreshments, discussing concrete development techniques and evaluation methods.

Toward a Place Where Industrial AI Developers Can Share Knowledge Across Industries

Construction and manufacturing.

The target workflows, the data used, and the shapes of the models developed all differ. And yet the two companies' development efforts had much in common.

How to create specialized data that barely exists in the world
How to structure the tacit knowledge of domain experts
How to design evaluation benchmarks that hold up in the field
How to balance model accuracy with inference cost
How to detect errors in the evaluation environment and the data itself

These challenges are not limited to construction and manufacturing.

They are themes common to many industries that deal with specialized knowledge — healthcare, logistics, energy, finance, agriculture, and more.

At the same time, how each company overcomes them strongly reflects the nature of the work and data it handles, and the philosophy of the engineers driving the development.

Through this event, we were reminded of the value of developers from different industries sharing not just success stories, but failures and trial and error as well.

At the networking session, we received comments like these from participants and sponsoring companies.

"Opportunities to hear about real development work at this level of technical detail are rare."
"It would be exciting if this grew into an ongoing community around the theme of specific industries × AI."

At Airion, we do not want this event to end as a one-off. We hope to expand it into a place where developers and companies working on industrial AI — beyond the bounds of manufacturing — can share their knowledge.

Rather than competing solely on AI model performance, the question is how to convert field knowledge into data, evaluate it, and deploy it safely into society.

We believe that discussion across industry and company boundaries will become increasingly important for the advancement of industrial AI.

Thank you to everyone who joined us.

Acknowledgments

We extend our heartfelt thanks to everyone at ONESTRUCTION Inc. for co-planning and co-hosting this event, to Mr. Kamata of Weights & Biases for sponsoring the food and introducing the latest technical developments, and to everyone who took time out of their busy schedules to attend.

We are also grateful to the panelists: Mr. Hidaka and Mr. Kanazawa of ONESTRUCTION, and Yoshihira and Nomura of Airion.

Airion will continue to share knowledge on AI development in manufacturing and other industrial domains, and to create opportunities for technical exchange across industries.