Top > IJAT > Development and Application of Breakdown T&BRL, a Novel Evalua ...

single-au.php

IJAT Vol.20 No.1 pp. 4-23
(2026)
doi: 10.20965/ijat.2026.p0004

Research Paper:

Views over last 60 days: 328

Development and Application of Breakdown T&BRL, a Novel Evaluation Framework for Social Implementation Assessment: A Case Study on Mobility Robot Devices

Gen Kudo*,† ORCID Icon, Andante Hadi Pandyaswargo** ORCID Icon, and Hiroshi Onoda* ORCID Icon

*Graduate School of Environment and Energy Engineering, Waseda University
513 Waseda-tsurumaki-cho, Shinjuku-ku, Tokyo 162-0041, Japan

†Corresponding author

**Environmental Research Institute, Waseda University
Tokyo, Japan

Received:
July 9, 2025
Accepted:
November 7, 2025
Published:
January 5, 2026
Creative Commons License
Keywords:
TRL, social implementation, innovation feasibility, readiness assessment, mobility robot
Abstract

This study introduces Breakdown T&BRL, an advanced evaluation method that extends traditional technology readiness levels (TRL) by incorporating four additional axes: business readiness level (BRL), social readiness level (SRL), governance readiness level (GRL), and human resource readiness level (HRL). This approach subdivides TRL/BRL 5 to 7 demonstration stages to precisely identify barriers to real-world application. In addition, this approach integrates evaluations from both developers and independent third parties to highlight critical discrepancies and inform tailored roadmaps. This study applied Breakdown T&BRL to a case study on next-generation mobility robot device development within the Minami-Kurihashi bridge life platform (BLP) concept. The results of the case study demonstrated its effectiveness in pinpointing specific issues for social implementation, including the need for production-ready products and sustainable business models, while also proposing concrete countermeasures. The comparative analysis with previous Breakdown T&BRL case studies revealed unique trends for this mobility project. The TRL/BRL evaluation gap between developers and third parties was small, which can be attributed to consistent information sharing from extensive BLP events and demonstrations. Third-party SRL and GRL evaluations were notably high, which suggests that proactive stakeholder engagement through consortium operations and discussions with regulators significantly enhanced social acceptance and regulatory alignment for these mobility devices. These findings highlight how comprehensive engagement can notably influence technology’s successful path toward social implementation.

Close full text
Cite this article as:
G. Kudo, A. Pandyaswargo, and H. Onoda, “Development and Application of Breakdown T&BRL, a Novel Evaluation Framework for Social Implementation Assessment: A Case Study on Mobility Robot Devices,” Int. J. Automation Technol., Vol.20 No.1, pp. 4-23, 2026.
Data files:

1. Introduction

Technology readiness assessment (TRA), which is also known as technology assessment (TA), originated as a method to manage technical risks in large-scale projects, particularly within space exploration and military contexts 1. At its core is the technology readiness level (TRL), which is an indicator introduced by the U.S. National Aeronautics and Space Administration in the 1970s for the objective evaluation of technology 2.

The initial seven-stage model encountered difficulties in terms of misunderstandings between researchers and management, especially at the higher levels of interpretation. Therefore, in 1995, it was expanded and refined into a nine-stage framework covering everything from basic research to final demonstration 3. Since the 1990s, TRL has been adopted by the U.S. Department of Defense (DoD) and the U.S. Department of Energy to meet their own requirements 4,5. In the 1990s, the European Commission and European Parliament adopted TA or similar concepts 6. In 2013, the International Organization for Standardization standardized TRL, establishing it as a global standard 7.

In Japan, a Japanese version of the TRA was developed based on the TRA developed in the United States. This Japanese standard was introduced by the Ministry of the Environment (MOE) and the Cabinet Office’s Cross-ministerial Strategic Innovation Promotion Program and is used as an important criterion for risk management and evaluation of technological development 8,9,10. Thus, TA has been widely used to evaluate the potential impact of specific technologies on society and support decision making 11.

The remainder of this paper is organized as follows. Section 2 provides a review of previous research and an overview of breakdown technology and business readiness level (T&BRL), which is the new evaluation method developed in previous research. Breakdown T&BRL evaluates the potential for social implementation and future innovation in research technologies in the carbon-neutral field 12. Section 3 describes the development of next-generation mobility robot devices, which are considered as the subject of the case study. Actual research technologies were also evaluated as case studies using Breakdown T&BRL. Section 4 presents the results of the case studies, and Section 5 discusses if Breakdown T&BRL can address and clarify the issues and limitations of evaluation using TRL alone. Finally, Section 6 summarizes the results of this study, clarifies the remaining issues, and discusses the limitations.

2. Related Work and New Evaluation Framework

2.1. TRL Issues and Limitations

2.1.1. Issues and Limitations of TRL in Previous Studies

TRL is not a perfect tool, and its limitations have been identified in previous studies. For example, TRL is limited to the operational environment and product system. Although it evaluates hardware and software, TRL does not represent the readiness to integrate technology 4. Further, TRL is insufficient for evaluating compatibility with human resources 13, and there is a problem with calculating the overall TRL of an integrated system with different levels of parts and applications 1. The current TRL model may need to be modified or supplemented because of its complexity in terms of innovation development 14.

The European Association of Research and Technology Organizations has identified four challenges that must be addressed when using TRL 15:

  1. (1)

    Lack of attention to setbacks in technology maturity: The linear TRL assessment model cannot adequately represent the nonlinear nature of research and technological development where trial and error and regressions occur 15. One example is when a technology assessed to be in the pilot production stage (TRL 5–6) regresses to the technical feasibility stage (TRL 2–3) because a design flaw makes it impossible to produce 14.
  2. (2)

    TRL is a single technology maturity approach: TRL is designed to assess individual component technologies and not the maturity of an entire system. This makes it difficult to assess complex technology-integrated projects 15.
  3. (3)

    TRL focuses on product development rather than manufacturability, commercialization, and organizational changes: This implies that TRL is essentially product-oriented and does not incorporate non-technical aspects such as social acceptance into its assessment criteria 15.
  4. (4)

    The context specificity of TRL scales: This leads to the risk of oversimplification and generalization. Different assessment objectives require different specifications and procedures, and therefore, the TRL scale must be optimized for the specific objectives of each organization 15.

2.1.2. Issues and Limitations of TRL Clarified Through Field Experience

Environmental issues such as global warming can be viewed as significant business opportunities. Meanwhile, research and technological development, social implementation, and innovation are considered urgent issues. Extensive investments have been made towards developing promising technologies, and research and technological development continue to progress. However, many resulting technologies are yet to be implemented in society 16. For example, in fuel cell technology, there is no established methodology to evaluate various competing technological developments and identify how results can be used in subsequent technologies 17.

Technology evaluation must be conducted to clarify multiple aspects of the target technology or system and verify its performance after social implementation 18. Although progress in research and technological development and an increase in the TRL scale are essential for realizing social implementation and innovation, the social implementation may not progress even with an increase in the TRL scale 10. This implies that TRL is not a perfect indicator to evaluate the possibility of the social implementation of technology and future innovation, and it is insufficient.

In addition to the four issues associated with TRL that must be addressed, we extracted four other issues based on our experience and knowledge from past projects in university laboratories, industry–academia collaboration at the Waseda University Incubation Center, and commercialization of research technology seeds. These issues are listed below:

  1. (1)

    Progress in TRL does not extend to social implementation, which suggests that factors besides TRL may affect the transition to social implementation or that TRL alone cannot adequately evaluate all elements necessary for successful implementation.
  2. (2)

    Developers tend to rate things highly, whereas third-party ratings are low. Researchers and developers tend to be overly optimistic when evaluating their own technologies. In contrast, third parties tend to aim for objective and evidence-based conclusions. However, external evaluations by third parties face several practical constraints. This discrepancy makes it difficult to accurately evaluate technologies and can significantly hinder their subsequent implementation in society.
  3. (3)

    Although conducting testing and demonstration experiments is easy, there is a significant barrier (TRL 5–7). Technological developments up to the laboratory testing and demonstration stages can be achieved relatively easily. However, an extremely large barrier appears between TRL 5 and 7, which is known as the “valley of death” in business management. This is a decisive turning point in the transition from development to social implementation 19.
  4. (4)

    TRL indicates the state of technological readiness. Researchers, engineers, and businesses need to identify existing bottlenecks and understand the methods, sequences, and costs required to resolve these obstacles toward social implementation.

2.2. Previous Attempts in Improving TRL

Several efforts have been made to address issues and limitations of TRL, which includes refining its definition and description of the TRL scale. For example, the linear approach of TRL is incompatible with the iterative nature of research and development (R&D), and suggestions for improving it have been proposed, including redirecting stalled technologies to a different development path or adding a co-creation step 14.

For the problem of the “single technology maturity approach,” TRL evaluates individual technologies instead of complex systems. For manufacturing aircraft and vehicles, TRL 6 is an important stage that indicates the readiness to consider the application of new technologies. Although TRL 6 is necessary, it is insufficient to evaluate the feasibility of applying the technology to the platform 20. The DoD developed the integration readiness level (IRL) and manufacturing readiness level (MRL). In 2006, the system readiness level methodology was proposed by the Systems Development and Maturity Laboratory at Stevens Institute of Technology to fill the gap between TRL and IRL 20. These readiness levels (RL) are combined into the total TRL, which is the result of TRL evolution over time to fill the gap from discovery to feasibility, practicality, and applicability 20. For the argument of “focus on product development rather than manufacturability, commercialization, and organizational changes,” TRL is aimed at product-oriented technologies. Therefore, even if it can explain the producer innovation model, it cannot fully explain the innovation model including modern users and open collaboration models 14.

In Japan, the Strategic Innovation Program (SIP) determined that TRL is insufficient as an evaluation indicator for social implementation and therefore adopted a multifaceted evaluation method that involves introducing a roadmap that considers five perspectives: technology, business, social acceptability, governance, and human resources 10. Going beyond the scope of R&D and achieving social acceptance and transition is essential for achieving social implementation and innovation. A multifaceted evaluation of SIP responds to this broader demand. The diffusion of innovation requires early adopters, who are opinion leaders in society. This requires communication, not just commercialization, through R&D progress 21.

From the perspective of socio-technical transitions, technological transitions involve changes not only in technology but also in user practices, regulation, industrial networks, infrastructure, and symbolic meaning or culture, and the innovation may be hindered by unsustainable existing systems known as regimes and their associated rules and institutions 22,23. Given this perspective, the multidimensional evaluation method suggested by SIP is appropriate to evaluate the potential for future innovation.

For the “context specificity of TRL scales,” conventional TRL levels were determined by relatively abstract definitions, and therefore, they often tend to be subjective and qualitative. This indicates that the evaluation criteria are vague and the conclusions may differ between different evaluators and evaluation processes. Such ambiguity hinders effective communication and decision-making among stakeholders 15.

2.3. Attempts for TRL Improvements in the Carbon Neutrality Field

Currently, efforts are underway to improve the application of TRL in the field of carbon neutrality 24. Evaluations based solely on experience can lead to subjective judgments, and therefore, a three-step systematic approach (identifying technological elements, evaluating the TRL of individual elements, and planning a maturation strategy) is recommended for ensuring consistency in evaluations 24. The MOE developed and introduced a TRL calculation tool for objectively evaluating research eligible for subsidies 9.

In the discussion regarding the TRL scale, some argued that it is too detailed, and others argued that it is too general. Ogawa 25 found that, in the field of waste management, the existing TRL axis is insufficient for precise evaluation. For example, technology that has reached TRL 9 in the logistics field may reach only TRL 7 in the waste management field 25. Further, evaluating systems that combine multiple technologies such as artificial intelligence (AI) and the Internet of Things (IoT) is challenging because the TRL of individual components can differ significantly. A new evaluation axis, “technology and system readiness level for waste management (T&SRL for WM),” was proposed to address this issue 25.

Kanao 26 reported that, in the development of an automated delivery robot, evaluations performed using the axis from a previous study 27, TRL, social readiness level (SRL), organizational readiness level, and legal readiness level resulted in higher evaluations than the actual situation, and detailed and specific condition settings were required. Further, problems arose where the improvement in the maturity of a certain perspective affected multiple maturity levels, thereby resulting in ambiguity in the evaluation. Consequently, the evaluation axis was integrated, standardized, and redefined as “T&SRL,” integrating TRL and SRL, to provide clearer guidelines 26.

Further, the authors developed a new maturity assessment method for carbon-neutral technologies, referred to as Breakdown T&BRL. We applied Breakdown T&BRL to a technology called carbon dioxide capture fuel recycle and verified its effectiveness in a case study 28.

figure

Fig. 1. Development of break-down T&BRL (source: 12).

2.4. Breakdown T&BRL

In our previous study, we developed Breakdown T&BRL as a new technology evaluation method 12. To this end, we clarified the issues and limitations of existing TRL based on a literature survey and organized measures for improving evaluation methods used in previous research. Subsequently, we used our experience and knowledge gained from the research, development, and social implementation of university-based technologies, including industry–academia collaboration and business incubation. Finally, we provided an overview of previous attempts for expanding and improving TRL and developing Breakdown T&BRL as a new evaluation method to solve these problems. Fig. 1 shows the development of Breakdown T&BRL in our previous study, which has four primary features 12:

  1. (1)

    Expansion of the assessment axis: Breakdown T&BRL uses not only TRL but also the other four assessment axes used in SIP, namely, TRL, BRL, SRL, governance readiness level (GRL), and human resource readiness level (HRL). This multidimensional approach helps identify various barriers to social implementation that are often overlooked by TRL 10.
  2. (2)

    Subdivision of the demonstration stage (TRL/BRL 5 to 7): This stage is a crucial barrier (hurdle) for commercializing many research technologies 19. Subdividing TRL and BRL makes it possible to identify bottlenecks in commercialization more clearly. The SRL, GRL, and HRL are evaluated qualitatively.
  3. (3)

    Multifaceted evaluation perspective: Breakdown T&BRL incorporates evaluations from both the developer and other stakeholders (e.g., users, customers, and third parties). Highlighting the discrepancy between the evaluations of both parties helps clarify problems that arise, particularly the tendency of developers to overestimate their own technology.
  4. (4)

    Practical solutions and road maps are presented. Breakdown T&BRL goes beyond simple technology evaluations to provide solutions for identifying bottlenecks and roadmaps for commercialization, thereby meeting the practical needs of researchers, engineers, and businesses.
figure

Fig. 2. Methodology framework of the case study.

3. Methodology of Case Study

3.1. Framework

In this study, we applied Breakdown T&BRL to an actual research technology (development of mobility robot devices) as a case study. Data collection for the case study was conducted through interviews with technology developers and other stakeholders. Based on the results, we consider if the issues and limitations of TRL identified thus far have been resolved by Breakdown T&BRL. In addition, we compare the status of other developments in mobility robot devices and consider the usefulness of this method for Breakdown T&BRL. Fig. 2 illustrates the case study framework.

3.2. Case Study: Development of Next-Generation Mobility Devices

The development of next-generation mobility robot devices for the Minami-Kurihashi bridge life platform (BLP) project is the subject of research in this study. This project is currently performing technology development from demonstration experiments to social implementation, and it is easy for conducting questionnaires and interviews with developers and stakeholders. Therefore, it is appropriate to verify the technology evaluation and its usefulness using Breakdown T&BRL.

Among mobility devices, we focused on the type of mobility device that can be driven autonomously and controlled remotely. This has many similarities to technological developments in other projects related to autonomous delivery robots and autonomous driving mobility we are involved in, which makes it easy to draw a comparison. In the case study of this research, we evaluated the project using Breakdown T&BRL, identified bottlenecks, and presented countermeasures and a roadmap. Subsequently, it is possible to evaluate the appropriateness of the results by comparing them with those of other projects.

3.2.1. Development of Mobility Devices in BLP

A project proposing a new lifestyle, called the Minami-Kurihashi BLP concept, is underway in the block in front of Minami-Kurihashi Station in Kuki City, Saitama Prefecture. In response to the increased demand for suburban living caused by the promotion of remote work attributed to the spread of COVID-19, five government and academic parties (Kuki City, Tobu Railway Co., Ltd., Toyota Housing Corporation, Aeon Retail Co., Ltd., and Waseda University’s Onoda Laboratory) collaborated to promote greenfield-type next-generation urban development. One feature of this project is that it is possible to design roads with autonomous driving technology. Commercial facilities and residential blocks are adjacent, and it is easy to conduct demonstration experiments on autonomous delivery using mobility robots. Regular town opening events and information sessions were held, and Waseda University’s Onoda Laboratory conducted mobility test drives and demonstration experiments. Fig. 3 shows an overview of the BLP 29.

Onoda Laboratory has developed and manufactured next-generation mobility devices that can be driven on public roads and conducted demonstration experiments every year in the Minami-Kurihashi BLP project. Fig. 4 shows the results of the development of multibenefit mobility in the BLP 29.

figure

Fig. 3. Overview of BLP (source: 29).

figure

Fig. 4. Results of developing multi-benefit mobility in BLP (source: 29).

In the BLP, Waseda University’s Onoda Laboratory conducted proof-of-concept (PoC) experiments using various mobility robots, including functions such as the human-following mode, remote operation, and autonomous driving. The human-following prototype used a stereo camera, and AI was utilized as a “mobile trash can” at the May 2022 BLP event. This has been deployed for event waste collection in specific areas such as parks, where road traffic laws do not apply. The waste collection operation was completed without any issues under the supervision of two to three monitors. The remote operation system compliant with the revised Road Traffic Act for “remotely operated small vehicles” was built. This enables Internet-based remote control via three webcams. A PoC in the BLP district in June 2023 faced image transmission delays because of poor communication; however, reproducibility was confirmed under stable communication in other regions. Plans include the local 5th generation, and we plan limited route tests from commercial facilities to residential areas 29.

For the multi-benefit trailer, the project initially developed a prototype robot for non-contact waste bin transportation; however, it lacked autonomous driving capabilities. This prototype was designed to maneuver beneath a waste bin (78 L with a dolly) using gripping arms for securing and towing it, and they could be remotely controlled via a remote. Through practical driving experiments, we confirmed that non-contact waste collection was feasible by having the robot tow a dolly-equipped waste bin. However, the design of the robot, which required it to maneuver underneath the bin, imposed height constraints that limited the integration of additional functionalities. This made it challenging to incorporate autonomous driving features such as light detection and ranging (LiDAR). Consequently, we relocated the towing mechanism to the rear of the robot, eliminating the height constraint that enabled installing essential autonomous driving equipment such as LiDAR. Based on these findings, we determined that an autonomous robot should incorporate four key functions for an effective operation: cruising, coupling with a dolly equipped waste bin, inserting the waste bin into a designated waste station, and waiting at a target location. In addition, the robot must possess the following capabilities 30:

  • Automatic coupling with a dolly-equipped waste bin and the automatic insertion of a waste bin into a waste station.

  • Autonomous navigation while towing a dolly, thereby avoiding obstacles such as people and vehicles.

  • Compatibility with outdoor flat asphalt and interlocking block paved surfaces.

  • Sufficient size to navigate paths with a width of 2 m.

  • Operation at speeds of 1.7 m/s (6 km/h) or less in accordance with the Road Traffic Act.

Based on these requirements, the project developed an autonomous robot along with a dolly that can carry waste bins of up to 108 L and a waste station frame. The dimensions of the robot were 650 mm (width), 1500 mm (length), and 810 mm (height). The robot can tow objects weighing \(\sim\)150 kg on a flat terrain at an inclination of 2° or less. The robot specifications included a three-dimensional (3D) LiDAR mounted on the upper front and two-dimensional (2D) LiDAR mounted on the lower front. Further, it employs a rear wheel drive and achieves autonomous navigation based on information from LiDAR units. The rear towing mechanism enables the gripping and release of the dolly handle with precise angle fixation for accurate alignment. An additional rear 2D LiDAR was installed at the bottom of the towing mechanism to detect obstacles. For cruising, the front 2D LiDAR acquires the surrounding information, which enables autonomous navigation while avoiding obstacles. The standard cruising speed is set to 0.5 m/s. A retroreflective material affixed to the top of the waste station was detected by 3D LiDAR to couple and insert waste bins into the waste station. This enabled the robot to accurately maneuver to the correct position for these operations. Although the current autonomous navigation methods rely on 2D LiDAR data, future improvements are planned to enable 3D LiDAR-based autonomous navigation for navigating inclines 30.

For autonomous navigation control, the robot generates a map of the surroundings using 2D LiDAR, which forms the basis for self-position estimation and path planning during autonomous driving. After map creation, various parameters such as target destinations, no-go areas, low-speed zones (0.1 m/s), and high-speed zones (0.7 m/s) can be customized to suit specific operational environments. The standardized movements of the robot (e.g., move, stop, couple, and insert) are controlled by commands written in a scenario file in text format.

For coupling and insertion control, the robot identified the central point of the retroreflective material on the waste station using 3D LiDAR accurately, which enabled precise positioning. After moving to an accurate position, the robot reverses towards the waste station and controls the ascent or descent of its claws. The numerical conditions must be adjusted based on the operational location, which can be modified by the robot operator as a variable. Currently, the objective is to complete the coupling and insertion operations within 100 s; the numerical conditions are currently being fine-tuned. For functional verification in a development environment, tests were conducted on May 2, 2023, on a flat asphalt-paved surface with an inclination of 1° or less within the Waseda University Honjo Campus. A waste collection command was issued after setting up one waste station at the end of a 5.0 m \(\times\) 15 m space and generating a 2D LiDAR map. The ability of the robot to autonomously collect a waste bin from a randomly set starting point and move to the target destination was verified. In addition, its ability to return the waste bin to the waste station and move it to a designated point was tested. The traffic cones were placed as obstacles along the route to verify the ability of the robot to navigate around them to the target destination. Consequently, the robot successfully performed the entire sequence of operations without colliding with any obstacles when operating at a normal speed of 0.5 m/s. A certain amount of space was required in front of the waste station for maneuvering to couple with the dolly equipped waste bin and insert it into the waste station. It was confirmed that the coupling and insertion functions could be performed normally if \(\sim\)3.0 m \(\times\) 3.0 m of the space could be provided. Autonomous driving was verified on a 2 m wide indoor flooring surface after map creation, which confirms that the robot could navigate without hitting the walls. Fig. 5 shows the development of autonomous robot mobility that can perform non-contact waste bin transportation 30.

figure

Fig. 5. Development of the autonomous robot mobility capable of non-contact waste bin transportation (source: 30).

The need to expedite the coupling and insertion operations of an autonomous driving robot was identified as a technical challenge from the demonstration experiments. Prolonged coupling and insertion times obstruct pedestrian and vehicular traffic, which necessitates a swift execution. Although the current target is to complete these operations within 100 s, the demonstrations on interlocking pavements revealed longer durations. This can be attributed to the unevenness of the pavement, which prevented the wheels from making proper contact with the ground, thereby causing the robot to repeatedly execute recovery maneuvers and fail to meet the predefined conditions.

Adjustments to waste station placement (e.g., asphalt paving with fewer irregularities) or modifications to the recovery operation settings are necessary to resolve this issue. Although increasing these settings reduces the number of recovery attempts, it compromises the precision of positional adjustment, increasing the probability of coupling and insertion failures. Thus, it is preferable to adjust the waste station placement. Considering the widespread adoption and utilization of this robot in diverse environments, it is necessary to identify and accumulate data on stable coupling and insertion settings across various road surface conditions 30.

BLP involves advancing service development based on the needs of the stakeholders and leveraging integrated commercial and residential areas. The key focus is meeting the confirmed demand for immediate on-demand delivery to residents. As a precursor to autonomous driving, five special small motorized bicycles are prepared for a sharing trial to address the short-distance mobility needs such as commuting to the Minami-Kurihashi Station. To this end, a multibenefit mobility station that provides solar power charging and serves as both a mobility device storage and teleworking space is developed. Currently, the optimal future placement of temporary installations is determined through ongoing trials 29.

figure

Fig. 6. Basic configuration of the autonomous delivery robot (source: 31).

3.2.2. Development of Automated Delivery Robots on a Public Road

We focused on the development of next-generation mobility devices for the BLP and aimed to confirm the usefulness of Breakdown T&BRL through case studies. Furthermore, we compared it with other cases of the development of delivery robot devices to consider if the Breakdown T&BRL evaluation in this case study was appropriate. This section provides an overview of other developments in mobility robot devices conducted at Waseda University’s Onoda Laboratory.

Kyocera Communication Systems Co., Ltd. (KCCS) and Waseda University’s Onoda Laboratory are developing public road mobility services using autonomous delivery robots. Our aim is to resolve regional and logistical challenges such as addressing labor shortages in last-mile delivery and supporting residents with limited access to shopping, thereby contributing to the future development of smart cities.

Autonomous delivery robots are broadly categorized into two categories: “low-speed small-sized” models that operate on sidewalks (maximum speed \(=6\) km/h, adhering to electric wheelchair standards and regulated as “remotely operated small vehicles” under the revised Road Traffic Act), and “medium-speed medium-sized” models designed for roadway operation (with a maximum speed of \(\sim\)15–25 km/h, currently under evaluation). Low-speed, small robots are suited for point-to-point deliveries, whereas the medium-speed, medium-sized robots are considered more appropriate for handling multiple items in parcel delivery services and online supermarkets. This project focused on developing medium-speed, medium-sized, and autonomous delivery robots for roadway operations. Their design concept emphasized unmanned operation, traversal within designated road areas at speeds comparable to those of bicycles (15 km/h or less), and a simplified, highly efficient structure attributed to the absence of seats and climate control systems. The hardware is built on compact EV components and includes autonomous driving modules such as LiDAR, optical cameras, and global navigation satellite system (GNSS) receivers. When equipped with cost-effective sensors that ensure sufficient short-range object detection, a re-evaluation of the sensor performance is necessary for environments with higher speeds. Fig. 6 shows the basic configuration of an autonomous delivery robot 31.

Autonomous delivery robots are controlled by a system known as “remote autonomous driving,” which involves autonomous navigation complemented by continuous human monitoring and intervention from a remote location. The quality of wireless communication is critical, and an algorithm for automatic deceleration and standby upon communication degradation has been implemented for mitigating unforeseen disconnections. Enhancing the autonomous driving ratio to achieve economic viability is essential for improving remote monitoring and operational performance and ensuring safety under unforeseen circumstances. Demonstrations in several areas have achieved autonomous driving across nearly the entire route, with current efforts focused on extending the continuous operation time and distance. Remote operation is reserved for exceptional circumstances because it tends to be slow because of the demanding requirements of high resolution and low latency, which poses an efficiency challenge 31.

This robot estimates its self-position by correlating pre-generated electronic maps with sensing data, and then, it plans autonomous routes to target destinations. Self-position estimation integrates data from LiDAR and GNSS receivers and is compensated for by an inertial measurement unit. Although positional accuracy maintains a lateral error of less than 3 cm, which is contingent on satellite acquisition, and it results in consistent robot trajectories and lateral errors (instabilities) are more frequently caused by environmental factors such as swaying vegetation caused by the automatic avoidance functionality of the robot. Safety features include an approach-stop (stopping if an object approaches within 2.8 m, which risks collision) and automatic evasion. The approach-stop function similar to an advanced emergency braking system prioritizes collision prevention with extensive testing focused on human subjects, given the operation of the robot near people. This includes a safety test simulating scenarios such as a jogger (2 m/s) suddenly entering the roadway 31.

Three technological challenges were identified:

  1. (1)

    Hesitation observed among cyclists in mixed-traffic environments: Medium-speed robots and bicycles operate at similar speeds. Observations have shown that cyclists hesitate about which side to pass because robots drive close to the left side of the road shoulder. Therefore, enhancing the perceptibility of robots and addressing abrupt stops caused by bicycle instability are critical for smooth mixed traffic.
  2. (2)

    Detection of left/right objects at intersections with speed differentials: This is a cost optimization challenge. Even with the low-speed driving concept in residential areas, routes can necessitate crossing arterial roads, which requires a balance between cost constraints and performance. Extending the intersection vehicle recognition range may require specialized sensors, such as millimeter-wave front cross-traffic alert sensors, depending on the cost.
  3. (3)

    Evasion of diverse cases of parked vehicles and obstacles: The variety of parked vehicles and obstacles in residential areas poses a challenge for continuous operation. The automatic evasion of all types of obstacles in residential areas where residents may work on roadways has not yet been fully achieved. In addition, securing safe stopping spaces for deliveries on roads occasionally requires human intervention 31.

The following insights were gained from multiple public road demonstration experiments. (1) For home delivery services in suburban residential areas, robots performed deliveries for parcel carriers and mobile sales of convenience store items, expanding to parcel collection and e-commerce deliveries. The roadway-operating medium-speed, medium-sized robot proved suitable, and the residents appreciated the convenience of receiving deliveries at their preferred times. (2) Inter-business sharing services in an industrial area indicated that a single autonomous delivery robot was shared among multiple businesses (parcel delivery, convenience stores, cleaning services, and factories) for transportation and collection. Multipurpose potential such as delivering food and beverages to workers was evaluated highly. However, integrating robots into existing business workflows requires time, and some opinions suggest that implementation should first target businesses with high transaction volumes such as retail businesses. Challenges concerning the visibility of low-speed vehicles in mixed traffic with large vehicles were identified. (3) For achieving mobile sales services at destinations in residential areas with multiple condominiums near a city, a refrigerated and payment-enabled autonomous delivery robot performed mobile sales in parks and private condominium properties, targeting residents with shopping difficulties. Challenges to autonomous driving have become apparent, including the need to account for other vehicles on one-way circular roads and numerous vehicles parked by visitors or deliveries 31.

3.3. Data Collection

Two developers of next-generation mobility devices from the Onoda Laboratory at Waseda University participated in the study and were selected as developers. These evaluators are appropriate because they are aware of not only past technological developments but also the challenges, barriers, and hurdles for future social implementation. Third-party stakeholders included one employee from Kuki City, Saitama Prefecture; one from Toyota Home; and one other from the Honjo Waseda Research Park Public Interest Incorporated Foundation. These evaluators are appropriate because they have a certain degree of knowledge regarding next-generation mobility device development in the BLP and can perform objective evaluations. One of the aims of this study is to clarify the evaluation gaps between such evaluators.

The authors did not directly evaluate the next-generation mobility devices themselves and were instead involved in managing the series of evaluation processes, collecting data (interviews), supporting the evaluation using Breakdown T&BRL, organizing the collected data, and analyzing the evaluation gaps between developers and third parties. Further, we formulated measures and roadmaps for the challenges of next-generation mobility development in the Minami-Kurihashi BLP concept clarified by technical evaluation using the Breakdown T&BRL.

Data were collected through interviews with the authors, developers, and third parties. TRL and BRL were based on the Breakdown T&BRL scale. The following questions were used with reference to the definitions in the SIP and TRL calculation tool user manual (third edition) 9,10:

  • At what TRL/BRL level do you think the next-generation mobility development in the BLP is?

  • Why do you think it is at that level?

  • Why do you think it exceeds the previous level, or do you not reach the next level?

  • What are the reasons or evidence that support your assessment?

  • From a TRL/BRL perspective, are there any challenges to promote the social implementation of next-generation mobility development in the Minami-Kurihashi BLP project?

  • What are the specific challenges and how difficult are they?

  • What kind of changes are required to solve these challenges?

  • What methods can be used to effect such changes?

Table 1. TRL scale in Breakdown T&BRL (source: 12).

figure

Table 2. BRL scale in Breakdown T&BRL (source: 12).

figure

For SRL, GRL, and HRL, a qualitative evaluation was performed using the following questions:

  • What is the status of SRL/GRL/HRL for next-generation mobility development in the Minami-Kurihashi BLP project?

  • Why do you think it is in that status?

  • From the perspective of the SRL/GRL/HRL, are there any issues in promoting the social implementation of next-generation mobility development in the Minami-Kurihashi BLP project?

  • What are the specific issues and how difficult are they?

  • What type of changes are required to solve the problems?

  • What are the methods for bringing about such changes?

Tables 1–3 show the TRL scale in Breakdown T&BRL, BRL scale, and subdivided T&BRL scales, respectively.

3.4. Data Analysis

Based on the collected data, the next-generation mobility robot device development in BLP was evaluated using Breakdown T&BRL by dividing it into the perspectives of developers and third parties. For the evaluation axis, TRL and BRL were quantitatively evaluated according to the level of subdivision into which they fell, and SRL, GRL, and HRL were evaluated qualitatively. After evaluating each RL, we focused on the evaluation gap between developers and third parties and clarified the issues and bottlenecks in social implementation. Table 4 shows the evaluation results of TRL and BRL, and Table 5 shows the evaluation results of the SRL, GRL, and HRL. The authors considered and organized countermeasures for the issues and bottlenecks identified in this manner. Table 6 shows the issues and countermeasures for next-generation mobility development in the Minami-Kurihashi BLP concept. Finally, a roadmap and support measures for social implementation and dissemination are summarized. Fig. 7 shows the roadmap and support measures.

Table 3. T&BRL scale in Breakdown T&BRL (Source: 12).

figure

Table 4. TRL and BRL evaluation results, gaps, and bottlenecks for practical application.

figure

Table 5. SRL, GRL, and HRL evaluation results, gaps, and bottlenecks for practical application.

figure

4. Results

In this section, we describe how Breakdown T&BRL addresses and clarifies the issues and limitations of technology evaluation using only TRL. With respect to the “lack of attention to setbacks in technology maturity,” Table 6 shows how to improve each item of Breakdown T&BRL for realizing social implementation. Further, regarding the “single technology maturity approach,” the roadmap in Fig. 7 shows how to improve each evaluation axis to increase the possibility of social implementation of the entire system. With respect to the “focus on product development rather than manufacturability, commercialization, and organizational changes,” Tables 4 and 5 realize a multidimensional evaluation using five evaluation axes (TRL, BRL, SRL, GRL, and HRL) instead of using only TRL.

With respect to the “context specificity of TRL scales,” Breakdown T&BRL enables a more objective evaluation by incorporating assessments conducted by both developers and third parties. Regarding the observation that “progress in TRL does not necessarily lead to social implementation,” Table 6 shows how improvements across each item of Breakdown T&BRL increase the likelihood of social implementation. With respect to the tendency that “developers tend to rate technologies highly, whereas third-party ratings are lower,” Tables 4 and 5 illustrate the gap between subjective evaluations by developers and objective evaluations by third parties, which may impede social implementation and dissemination.

With respect to the observation that “testing and demonstration experiments can progress easily, whereas a major barrier exists between TRL 5 and TRL 7,” Tables 4 and 5 clarify the challenges and bottlenecks encountered when transitioning from demonstration testing to social implementation, as well as their countermeasures, by subdividing TRL/BRL 5 into seven stages.

With respect to the limitation that “TRL is a tool for indicating the state of technology readiness,” Breakdown T&BRL provides additional guidance beyond technical evaluation. Fig. 7 presents a roadmap and support measures for the social implementation and dissemination of next-generation mobility development within the Minami-Kurihashi BLP concept. This framework extends beyond technical readiness to illustrate how researchers, engineers, businesses, and other stakeholders can proceed toward social implementation.

Table 7 illustrates the eight challenges of TRL and how they were resolved by Breakdown T&BRL.

5. Discussion

This study aimed to confirm the usefulness of Breakdown T&BRL through case studies. Breakdown T&BRL has been confirmed to be highly useful compared to conventional evaluation methods that rely solely on TRL, which overcomes the issues and limitations of such methods. Its usefulness is based on the following three points.

Identifying issues and bottlenecks in social implementation and contributing to the formulation of appropriate improvement measures is essential by subdividing and carefully evaluating TRL/BRL 5–7 in the demonstration phase, which are the major challenges and hurdles to social implementation. Further, this approach contributes to the creation of a comprehensive roadmap that increases the possibilities of social implementation and subsequent innovation.

In the case study, the application of the subdivided T&BRL scale revealed issues and bottlenecks such as “necessary improvements being made to the product based on feedback from potential customers; however, it is still in the prototype stage,” “testing is being conducted by potential customers in a limited environment close to the production environment; however, full-scale validation has not been conducted,” and “the consideration and identification of management resources necessary for business expansion has not been completed,” “the development of a sustainable business model has not progressed,” and “a business plan that is highly feasible and at a level that will interest investors is yet to be developed” (BRL).

In response to these issues and bottlenecks, measures aimed at exceeding TRL 5 were derived, including “identify issues leading to full-scale mass production,” “improve the product for the production environment and ensure drivability and safety,” “test in the production environment, consider mass production, and build a manufacturing system,” and support measures such as “support demonstrations on public roads, search for and introduce mass production manufacturing bases.” Countermeasures for exceeding BRL include “identifying the necessary management resources for business expansion,” “support for formulating business reforms, including production and sales systems,” “establishing business entities and raising funds,” and support measures such as “support for formulating business reforms that are specific, have growth potential, and are highly feasible, and obtaining grants and subsidies from the Ministry of Economy, Trade and Industry and the Ministry of Land, Infrastructure, Transport and Tourism” were derived. Such detailed problem identification and presentation of solutions is a unique feature of this method and addresses the issues and limitations of conventional evaluations such as the so-called “valley of death” of TRL/BRL 5 to 7: “lack of attention to setbacks in technology maturity” and “developers tend to rate things highly, but third-party ratings are low.”

Breakdown T&BRL presents a method for evaluating technologies in a multidimensional manner using five evaluation axes (TRL, BRL, SRL, GRL, and HRL) and promoting social implementation not only of individual elements but also of the system as a whole. This not only improves the TRL but also enables the visualization of cases wherein social acceptability remains low even when the TRL is high (from the perspective of SRL) and wherein future legal regulations may be a barrier (from the perspective of GRL). It also suggests improvement measures according to the situation such as responding to social concerns and preparing ahead for regulatory changes. This systematic evaluation approach overcomes the issues and limitations of technology evaluation using the TRL method alone, that is, “lack of attention to setbacks in technology maturity,” “developers tend to rate things highly, but third-party ratings are low,” and “TRL is a tool that indicates the state of technology readiness.” In the case study, it was confirmed that there were no fatal issues or bottlenecks in the SRL, GRL, or HRL. However, in SRL, future measures such as “formulating and implementing popularization standards” and support measures such as “listing the elements and issues necessary for popularization and creating a popularization plan” and “supporting popularization efforts” were presented. In GRL, measures such as “deregulation” and “comparison with other technologies and products” were presented, and concrete support measures such as “encourage the police and the Ministry of Land, Infrastructure, Transport and Tourism to deregulate, and conduct demonstration experiments to obtain further driving records” and “demonstrate superiority through research and papers comparing with other mobility technologies.” In HRL, support measures such as “before the final stage of practical application and social implementation, confirm the procurement method in the human resources market and the training method” and “by advancing automation such as control, the necessary skill set will be reduced” were presented.

Table 6. Issues and countermeasures.

figure
figure

Fig. 7. Roadmap and support measures.

Table 7. How Breakdown T&BRL addressed and clarified the issues and limitations of TRL.

figure

Table 8. Difference between the result of this study and the previous study.

figure

Breakdown T&BRL significantly increases the objectivity of evaluation by imposing individual evaluations by developers (or researchers) and independent third parties (e.g., customers, experts, and external organizations).

Although no significant gap was observed between developers and third parties in the case study, this multifaceted approach enabled a more balanced and objective evaluation. Clarifying the gap in views through such a multifaceted evaluation provides concrete suggestions for solving problems and directly addresses the challenges and limitations of TRL-only evaluation, such as “context specificity of TRL scales,” “developers tend to rate things highly, but third-party ratings are low,” and “it is easy to test progress and perform demonstration experiments; however, there is a big barrier ahead (TRL 5 to 7).”

When comparing this study with the authors’ previous research, which is a case study of another technology evaluation using Breakdown T&BRL, tendencies were observed, such as “the TRL/BRL evaluation gap between developers and third parties is small,” “third parties’ SRL evaluations are high,” and “third parties’ GRL evaluations are high” 31. For the aspect that “there is a small TRL/BRL evaluation gap between developers and third parties,” this is attributed to the fact that the next-generation mobility development in BLP, which is the subject of this study, has already been carried out through several events, explanatory meetings, and demonstration experiments. Information sharing and awareness among stakeholders is consistent. For the aspect that “third-party SRL evaluation is high,” this is believed to be caused by acceptance among stakeholders increasing through events, explanatory meetings, and demonstration experiments, and the operation of BLP in a consortium format has effectively improved SRL. “Third-party GRL evaluation is high” because the BLP has held multiple discussions with regulators such as local governments and police and has been able to improve GRL (to raise TRL) in line with the progress of TRL. This indicates that improving TRL does not necessarily increase the possibility of social implementation. Paradoxically, these show that the social implementation possibility or future innovation possibility of a certain technology cannot be appropriately evaluated by TRL alone, which shows the usefulness of a multifaceted evaluation method such as the Breakdown T&BRL. This suggests that it is important not to aim to improve only the TRL but also consider which of the multiple axes to improve, in what order, and how to increase the possibility of the social implementation of a certain technology. For technologies such as mobility device development, including autonomous driving, which is the subject of this research, social implementation is likely to be difficult unless the SRL and GRL are improved regardless of the improvement in TRL. Conversely, mobility devices development in BLP was closer to social implementation improved the SRL and GRL in line with the improvement of the TRL. This shows that the technology evaluation by multiple axes in Breakdown T&BRL is not independent of each other, and instead, they are mutually related and affect each other. Table 8 shows the difference between the results of this study and those of a previous study 28.

Table 9. Readiness evaluation of BLP and KCCS.

figure

A comparison with other mobility robot development projects in which we were involved can provide suggestions for using the Breakdown T&BRL to advance the social implementation of technology. Yamamoto 32 conducted a maturity assessment of the multi-benefit trailer in the BLP and the autonomous driving robot by KCCS (demonstration experiment in Ishikari City) for fiscal years 2023 and 2024. Table 9 shows the readiness evaluation for each item.

The BRL of the multibenefit trailer in the BLP was not as advanced as that of the TRL. In this study, BRL and SRL were highly rated, which could be attributed to the understanding of stakeholders such as project participants and residents having progressed. Some mobility vehicles have become capable of driving on public roads, and business models are being considered based on use cases. In future, instead of developing our own technology and increasing only the TRL, we should incorporate already established technologies and products (for example, BLP should benchmark the autonomous robot developed by KCCS, which is already technologically advanced) and improve the five axes clarified in the Breakdown T&BRL in a balanced manner. However, the autonomous robot developed by the KCCS was highly evaluated for all items, and the BRL was considered to be well developed. The autonomous robot developed by KCCS is believed to have three major issues: improving autonomous driving technology, interface with the user (user interface (UI)/user experience (UX)), and linking with infrastructure and systems 31. KCCS has thoroughly examined the business feasibility, and therefore, we should solve the technical issues that came up in the demonstration experiment and improve the TRL to increase the possibility of social implementation of the system as a whole. Meanwhile, as in the BLP, we should make it possible to drive on public roads in consultation with local residents, local governments, and police and realize “safety and security,” improving the GRL and SRL.

6. Conclusions

The inadequacy of TRL for social implementation assessment has been identified because of its limitations such as neglecting non-technical aspects (social acceptance and business viability), difficulty in assessing complex systems, and inability to solve the “valley of death” (TRL 5–7). To resolve these issues, we developed Breakdown T&BRL, a novel framework that expands the evaluation axes to five axes (TRL, BRL, SRL, GRL, and HRL) in a previous study. The key features of T&BRL include subdividing TRL/BRL 5–7 to pinpoint bottlenecks and integrating evaluations from both developers and third parties to enhance objectivity and clarify assessment gaps.

In this study, we conducted a case study to apply Breakdown T&BRL to evaluate actual research technologies for overcoming the issues and limitations of TRL-only evaluation. Although we developed the Breakdown T&BRL to appropriately evaluate technologies in the carbon-neutral field, an important objective of this study was determining whether this evaluation method can be applied to other technologies (exemplified by mobility device development in this study). This study addresses this issue by interviewing developers and third-party stakeholders (local government and corporations) to collect data for the five readiness axes. The analysis focused on clarifying evaluation gaps and identifying specific barriers, which led to the formulation of countermeasures and a detailed roadmap. The study results showed that Breakdown T&BRL resolved the issues and limitations of TRL-only evaluation and was more useful than the existing technology evaluation methods. The application of the Breakdown T&BRL framework identified specific barriers to the social implementation of mobility devices, including the need for production-ready products (TRL 5-2) and a sustainable business model (BRL). The evaluations showed that proactive third-party stakeholder engagement leads to high SRL and GRL ratings. This confirmed the core utility of the framework: for autonomous mobility, the simultaneous improvement of non-technical axes (SRL/GRL) alongside TRL is critical for success, which makes the multidimensional Breakdown T&BRL essential for accurate assessment and practical guidance. The breakdown of the T&BRL is significant because it identifies the problems and bottlenecks faced by research technologies, formulates countermeasures and roadmaps, and concretely maps the path to social implementation and subsequent innovation.

Future challenges include expanding the scope of the application of Breakdown T&BRL and verifying its validity as an evaluation method for various research technologies. Breakdown T&BRL was introduced to improve the feasibility evaluation of social implementation and overcome the limitations of TRL-only evaluation; however, because of the nature of the case study method, it has not yet been demonstrated that it is a universally optimal approach for all research technologies.

The optimal evaluation technique and criteria may vary based on individual characteristics such as whether the target technology is software or hardware, whether the infrastructure of the existing technology can be used, and the degree of technical expertise. Further, appropriate indicators change depending on factors such as the progress of research and development, novelty of the technology, business model, and competitive environment. Therefore, it is an important verification issue in the future whether Breakdown T&BRL can continue to provide useful suggestions at all stages of social implementation of various technologies such as space development, robots, software, medicine, and drug discovery.

In future, the criteria for items evaluated qualitatively should be refined, and measures to resolve differences in views between evaluators should be explored. Research on categorizing evaluation items and more detailed subdivisions is also recommended. Approaches that use existing frameworks to analyze BRL from a different perspective or Breakdown SRL into social acceptability and market development levels can enable more objective and specific evaluations and contribute to identifying obstacles. Individual optimization of the evaluation criteria for a target technology through differentiation, subdivision, or integration of such evaluation axes is essential for accurately measuring the potential for future innovation, especially in nonlinear and complex R&D.

The continuous application of this method to various case studies in the future is expected to clarify potential issues and lead to iterative improvements that will improve the reliability of the evaluation tool. Breakdown T&BRL adopted five evaluation axes (TRL, BRL, SRL, GRL, and HRL). However, in future, other effective evaluation axes, subdivisions, integrations, or optimal elements and weightings will likely be found at the level of each evaluation axis. New insights will also be obtained from before and after the evaluation of measures for successful cases in past technological developments. Although single case studies have limitations in generalization, the methodology will be gradually improved through the expansion of the scope of application. This would contribute to improving the quality of outputs such as the formulation of more specific and detailed roadmaps and support measures, which are essential for practitioners seeking to increase the probability of social implementation.

Breakdown T&BRL offers remarkable advantages for various stakeholders. For technology developers, a design that separates the developer’s evaluation from that of a third party and focuses on the gap between them is extremely useful for accurately measuring the possibility of social implementation and suppressing self-overevaluation. Further, it provides an opportunity to objectively view the technology. Business operators can use this method to develop business plans based on issues in each evaluation axis, including the current TRL, to formulate business strategies to overcome issues and to determine the optimal timing and method of improvement. This is expected to contribute to the allocation of limited management resources to options such as technology development, customer development, and lobbying. For policy makers, it will provide an information base for assessing the need for deregulation and the direction of technology development support, such as subsidy provision. From the perspective of investors, a multidimensional analysis that goes beyond the perspective of TRL alone is useful for technological evaluation and important investment decisions. This is crucial in evaluating the likelihood of social implementation that leads to future capital gains. Finally, third parties such as evaluation agencies and analysts can use Breakdown T&BRL to ensure objective technology evaluation and convey the contents of that evaluation, benefitting from its detailed and mixed approach.

Acknowledgments

We would like to express our sincere gratitude to the members of BLP, Kuki City, Toyota Housing Corporation, and Honjo Waseda Research Park Public Interest Incorporated Foundation who cooperated in the interview surveys required for the writing of this paper. We would also like to thank the students of the Onoda Laboratory who provided advice and cooperation in data collection during this research.

References
  1. [1] M. Héder, “From NASA to EU: The evolution of the TRL scale in Public Sector Innovation,” Innov. J, Vol.22, Issue 2, Article No.3, 2017.
  2. [2] J. C. Mankins, “Technology readiness assessments: A retrospective,” Acta Astronaut., Vol.65, pp. 1216-1223, 2009. https://doi.org/10.1016/j.actaastro.2009.03.058
  3. [3] S. R. Sadin, F. P. Povinelli, and R. Rosen, “The NASA technology push towards future space mission systems,” Acta Astronaut., Vol.20, pp. 73-77, 1989. https://doi.org/10.1016/0094-5765(89)90054-4
  4. [4] G. Salazar and M. N. Russi-Vigoya, “Technology readiness level as the foundation of human readiness level,” Ergon. Des., Vol.29, Issue 4, pp. 25-29, 2021. https://doi.org/10.1177/10648046211020527
  5. [5] R. F. B. Santacruz, B. P. Sullivan, S. Terzi, and C. Sassanelli, “Developing a technology readiness level template for model-based design methods and tools in a collaborative environment,” Proc. of the IFIP Int. Conf. on Product Lifecycle Management, pp. 237-249, 2023. https://doi.org/10.1007/978-3-031-25182-5_24
  6. [6] M. D. Rogers, “Technology assessment in the EU institutions,” J. Disaster Res., Vol.6, No.5, pp. 522-527, 2011. https://doi.org/10.20965/jdr.2011.p0522
  7. [7] International Organization for Standardization, “Space systems – Definition of the technology readiness levels (TRLs) and their criteria of assessment,” ISO 16290:2013, 2013. https://www.iso.org/standard/56064.html [Accessed June 1, 2025]
  8. [8] I. Ihara, A. H. Pandyaswargo, and H. Onoda, “Development and the effective of the J-TRA: A methodology to assess energy technology R&D programs in Japan,” Proc. EcoDesign Products & Service Symp., pp. 109-117, 2018.
  9. [9] Ministry of the Environment of Japan, “TRL calculation tool user manual,” 2016 (in Japanese). https://www.env.go.jp/content/900443533.pdf [Accessed June 1, 2025]
  10. [10] Cabinet Office of Japan, “Cross-ministerial Strategic Innovation Promotion Program,” 2023 (in Japanese). https://www8.cao.go.jp/cstp/gaiyo/sip/sipgaiyou.pdf [Accessed June 1, 2025]
  11. [11] T. Suzuki and G. Yoshizawa, “Special issue on managing catastrophic technological risks and role of technology assessment (TA) in the post 3/11 society,” J. Disaster Res., Vol.6, No.5, pp. 473-475, 2011. https://doi.org/10.20965/jdr.2011.p0473
  12. [12] G. Kudo, A. H. Pandyaswargo, and H. Onoda, “Breakdown technology & business readiness level assessment: A novel evaluation framework for assessing technology social implementation and innovation potential in the field of carbon neutrality” Proc. 14th Int. Symp. on Environmentally Conscious Design and Inverse Manufacturing, 2025 (in press).
  13. [13] A. Olechowski, S. Eppinger, N. Joglekar, and K. Tomaschek, “Technology readiness levels: Shortcomings and improvement opportunities,” Syst. Eng., Vol.23, No.4, pp. 395-408, 2020. https://doi.org/10.1002/sys.21533
  14. [14] S. Yfanti and N. Sakkas, “Technology readiness levels (TRLs) in the era of co-creation,” Applied System Innovation, Vol.7, Issue 2, Article No.32, 2024. https://doi.org/10.3390/asi7020032
  15. [15] European Association of Research and Technology Organizations, “The TRL scale as a research & innovation policy tool, EARTO recommendations,” 2014. https://www.earto.eu/wp-content/uploads/The_TRL_Scale_as_a_R_I_Policy_Tool_-_EARTO_Recommendations_-_Final.pdf [Accessed June 1, 2025]
  16. [16] Cabinet Office of Japan, “Innovative environmental innovation strategy,” 2020 (in Japanese). https://www8.cao.go.jp/cstp/siryo/haihui048/siryo6-2.pdf [Accessed June 1, 2025]
  17. [17] C. Y. Huang, Y. H. Hung, and G. H. Tzeng, “Using hybrid MCDM methods to assess fuel cell technology for the next generation of hybrid power automobiles,” J. Adv. Comput. Intell. Intell. Inform., Vol.15, No.4, pp. 406-417, 2011. https://doi.org/10.20965/jaciii.2011.p0406
  18. [18] Y. Kikuchi, A. Heiho, Y. Dou, I. Suwa, I.-C. Chen, Y. Fukushima, and C. Tokoro, “Defining requirements on technology systems assessment from life cycle perspectives: Cases on recycling of photovoltaic and secondary batteries,” Int. J. Automation Technol., Vol.14, No.6, pp. 890-902, 2020. https://doi.org/10.20965/ijat.2020.p0890
  19. [19] T. Degawa, “The Concept of Technology Management from the Field of MOT and Development Ventures,” Kobunsha, 2004 (in Japanese).
  20. [20] J. Yu, R. Wahls, B. Esker, L. Lahey, D. Akiyama, M. Drake, and D. Christensen, “Total technology readiness level: Accelerating technology readiness for aircraft design,” Proc. Conf. AIAA Aviation 2021 Forum, 2021. https://doi.org/10.2514/6.2021-2454
  21. [21] E. M. Rogers, “Diffusion of Innovations,” Free Press, 1962.
  22. [22] F. W. Geels, “Technological transitions as evolutionary reconfiguration processes: A multi-level perspective and a case-study,” Res. Policy, Vol.31, Issues 8-9, pp. 1257-1274, 2002. https://doi.org/10.1016/S0048-7333(02)00062-8
  23. [23] F. W. Geels, “Regime resistance against low-carbon transitions: Introducing politics and power into the multi-level perspective,” Theory, Cult. & Soc., Vol.31, Issue 5, pp. 21-40, 2014. https://doi.org/10.1177/0263276414531627
  24. [24] A. W. Zimmermann, T. Langhorst, S. Moni, J. A. Schaidle, F. Besebaa, and A. Bradow, “Life-cycle and techno-economic assessment of early-stage carbon capture and utilization technologies – A discussion of current challenges and best practices,” Front. Clim., Vol.4, Article No.841907, 2022. https://doi.org/10.3389/fclim.2022.841907
  25. [25] A. Ogawa, “Research on the advancement of waste management systems and its evaluation using digital technology,” Ph.D. thesis, Waseda University, 2024 (in Japanese).
  26. [26] Y. Kanao, “Research on the development and evaluation of next-generation mobility devices, including automated delivery robots – Development towards the Bridge Life Platform concept in Minami-Kurihashi Station area,” Master’s thesis, Waseda University, 2023 (in Japanese).
  27. [27] I. Bruno, G. Lobo, B. Valente Covino, A. Donarelli, V. Marchetti, A. Schiavone Panni, and F. Molinari, “Technology readiness revisited: A proposal for extending the scope of impact assessment of European public services,” Proc. 13th Int. Conf. on Theory and Practice of Electronic Governance, pp. 369-380, 2020. https://doi.org/10.1145/3428502.3428552
  28. [28] G. Kudo, A. H. Pandyaswargo, and H. Onoda, “Application of breakdown technology & business readiness level assessment to carbon capture technology: A case study on advancing social implementation assessment,” 14th Int. Symp. on Environmentally Conscious Design and Inverse Manufacturing, 2025 (in press).
  29. [29] H. Onoda, “An attempt to implement multi-benefit mobility in society in conjunction with urban development: Taking the Minami-Kurihashi Station Block ‘BRIDGE LIFE Platform’ concept as an example,” J. Inf. Process. Soc. Jpn., Vol.65, No.5, pp. d1-d10, 2024 (in Japanese). https://doi.org/10.20729/00233636
  30. [30] A. Ogawa, K. Kubota, A. H. Pandyaswargo, R. Tsubouchi, K. Ozawa, M. Sawada, and H. Onoda, “Development of an autonomous robot for a contactless waste collection system and study for its social implementation,” IPSJ Trans., Vol.5, No.2, pp. 1-9, 2024 (in Japanese).
  31. [31] K. Masuda and H. Onoda, “Challenges for demonstrating and commercializing autonomous delivery robots that can be driven on public roads,” J. Inf. Process. Soc. Jpn., Vol.65, No.5, pp. d11-d22, 2024 (in Japanese). https://doi.org/10.20729/00233637
  32. [32] Y. Yamamoto, “A study on performance evaluation of autonomous delivery mobility in micro communities and the proposal of use cases for social implementation,” Master’s thesis, Waseda University, 2024 (in Japanese).

Creative Commons License  This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Apr. 03, 2026