Validating the Formal Specification of the THUNDERS Process

Validación de la especificación formal del proceso THUNDERS

Authors:
  • Vanessa Agredo-Delgado
  • Pablo H. Ruiz
  • Cesar A. Collazos
DOI 10.22430/22565337.2658 Logotecnologicas PDF Table Figure

Received: Febrero 20, 2022
Accepted: Agosto 23, 2023
Available: Agosto 29, 2023

 

How to cite / Cómo citar
V. Agredo-Delgado, P. H. Ruiz, C. A. Collazos, “Validating the formal specification of the THUNDERS process,” TecnoLógicas, vol. 26, nro. 57, e2658, 2023. https://doi.org/10.22430/22565337.2658

 

Highlights

  • THUNDERS builds shared understanding in problem-solving activities.
  • Shared understanding is the agreement of participants' perceptions on a subject.
  • The better the understanding and cohesion among team members, the better the results obtained.
    • Abstract

    Collaborative work encourages participants to build knowledge through exploration, discussion, negotiation, and debate to generate a better understanding or shared understanding of a concept, problem, or situation within a group. The aim of collaborative work is to find that shared understanding, which is understood as the existing agreement or similarity in the perceptions of the participants on a topic. Considering this, it can be determined that the greater the understanding and cohesion among all team members, the better results will be obtained in the development of the tasks and responsibilities that each of the members must fulfill, generating greater group trust and allowing everyone to move in the same direction. However, such understanding is not easy to build, there is no clarity on how it should be built, and it is simply given as something obvious or to be achieved, without giving it real importance. Trying to address these problems, from the multi-cycle action research methodology, THUNDERS is defined as a process that establishes how to build shared understanding in problem-solving activities. This article shows the conceptual and methodological cycles for its construction, and more in detail the validation cycle, in which was performed: an expert validation of the formal specification of THUNDERS to determine the correctness and completeness of its structure, a quality validation of its process model SPEM 2.0, and an experiment to validate its feasibility and usefulness. As results of these validations, it was obtained that THUNDERS needs to improve the syntactic and semantic elements of its specification and the cognitive load generated by its use. In addition, it was found that is a viable and useful process for the construction of a shared understanding with each of the elements that compose it.

    Keywords: Computer supported collaborative work, shared understanding, problem-solving activity, validation process, formal specification in SPEM 2.0.
    Resumen

    El trabajo colaborativo permite incitar a los participantes a la construcción de conocimiento mediante la exploración, discusión, negociación y debate, con el fin de generar una mejor comprensión o entendimiento compartido de un concepto, problema o situación dentro de un grupo. El objetivo del trabajo colaborativo es encontrar ese entendimiento compartido, el cual se entiende como la concordancia o similitud existente en las percepciones de los participantes sobre una temática. Considerando esto, se puede determinar que cuanto mayor sea el entendimiento y la cohesión entre todos los miembros del equipo, mejores resultados se obtendrán en el desarrollo de las tareas y responsabilidades que cada uno de los miembros debe cumplir, generando mayor confianza grupal y permitiendo que todos avancen en la misma dirección. Sin embargo, dicho entendimiento no es fácil de construir, no hay claridad en cómo se debe construir, y sencillamente se da como algo obvio o que debe lograrse, sin darle la real importancia. Tratando de abordar estas problemáticas, a partir de la metodología de investigación-acción multiciclo se define THUNDERS, un proceso que establece cómo construir entendimiento compartido en actividades de resolución de problemas. Este artículo muestra los ciclos conceptual y metodológico para su construcción, y más en detalle el ciclo de validación, en el cual se realizaron: una validación de expertos de la especificación formal de THUNDERS para determinar la correctitud y completitud de su estructura, una validación de calidad de su modelo de proceso SPEM 2.0, y un experimento para validar su viabilidad y utilidad. Como resultado de estas validaciones se obtuvo que THUNDERS necesita mejorar elementos sintácticos, semánticos de su especificación, y la carga cognitiva que genera su uso. Además, se obtuvo que es un proceso viable y útil para la construcción de un entendimiento compartido con cada uno de los elementos que lo componen.

    Palabras clave: Trabajo colaborativo asistido por computador, entendimiento compartido, actividad de resolución de problemas, proceso de validación, especificación formal en SPEM 2.0.

    1. INTRODUCTION

    The definition of collaborative work refers to, it is one in which a group of people contribute their ideas and knowledge to achieve a common goal, seeking the production of knowledge, unlike teamwork where the optimization of results is sought [1]. The so-called collaborative work is defined when team members convene a meeting to solve a problem, which in most cases is technical [2]. In this sense, collaboration involves direct interaction between individuals to produce a product and involves negotiations, discussions, and reaching a consensus on the perspectives or ideas of others to obtain the expected results [3]. In this sense, the interaction will allow true collaborative work to arise, and this will encourage the efforts of the group members to bear fruit, facilitating the success of each member to achieve the common goal [4]. This is where the importance of a quality dialogue (active listening and participation) is essential for the interaction in order to encourage understanding among the participants and therefore, the collaborative work [5]. One way to achieve this quality dialogue, is to seek the achievement of shared understanding (SU) among the participants of the collaborative activity [6], for its part, shared understanding, refers to agents constructing equivalent artificial information systems, as a result of individual perceptions and the flow of information [7], therefore, "is a result or consequence of information flow, and cannot be built without it [8]. Likewise, it refers to, the moment when members create a new joint perspective that emerges from their collective contributions" [9]. It is what happens when equivalent artificial information systems are constructed, and the corresponding mappings between these are also constructed by each agent engaged in a dialogue, enabling a group of agents to act in a coordinated manner so that they can collaborate with each other to achieve their individual objectives. SU is a prerequisite for collaborative work among many agents when two agents engage in a dialogue [8]. However, this SU can be difficult to achieve, because, in this dialogue among the participants, there are either lack of experience or different experiences, differences in knowledge, variety of contexts, and the actors’ language [10]. In addition, the complexity of considering all opinions, points of view, and, that everyone understands and agrees with the topic of the activity and with the main idea of what is going to be solved [11]. Different words are also used for the same meanings, or vice versa, which does not allow for good communication and different ideas and perceptions, which complicates the work as a whole [12].

    This paper is an extension of the work originally presented in [13], and specifically here, in the quest to solve some of these problems, the THUNDERS (collaboraTive work through sHared UNDErstanding in pRoblems-solving activitieS) process is explained in more detail, as well as the following validations: expert validation of the THUNDERS formal specification to determine the correctness and completeness of its structure, the quality validation of its SPEM 2.0 [14] using the AVISPA visual analysis method [15], and an experiment validating its feasibility and usability. THUNDERS allows to support in a guided way the design, execution, and verification of the solution of a problem, the achievement of shared understanding and performance of the participants in collaborative problem-solving activities, using a set of phases, activities, tasks, work products, roles, and workflows in a planned way. The multi-cycle action research methodology with bifurcation was used in the construction of the process [16]. According to the results of the validations, it can be said that THUNDERS needs to improve syntactic and semantic elements of its specification in SPEM, improve the cognitive load generated by its use, and it was also obtained that it is a viable and useful process for the construction of a shared understanding with each of the elements that compose it; likewise, the need to incorporate mechanisms that monitor and help the permanence of the understanding throughout the execution of the activity was identified.

    This paper is structured as follows: Section 2 contains related work. Section 3 contains the methodology for defining the THUNDERS process. Section 4 contains the conclusions and future work.

     

    2. RELATED WORKS

    2.1 Achieving shared understanding

    Granados in [17] analyzed the conceptual structure from the messages and types of messages that group members exchange while performing the task, defining that shared understanding is achieved with messages, clarifications type statements, as well as questions formulated within the group. Similarly, De Haan [18] was concerned not only with how groups define the object of the activity (the task model), but also with the so-called "team interaction model" in which divided roles and responsibilities are used to solve a particular task and analyze these interactions to achieve a shared understanding. On the other hand, in [19], they conceptualized shared understanding, applying collaborative engineering to derive a validated collaborative process module (using the thinkLet "MindMerger") to systematically support heterogeneous workgroups in building shared understanding. In [20] a model of the shared understanding building process was designed, which incorporates three main features: division of labor, coordinated perception, and mediated coupling. However, there is still a need for further research on how this process occurs and how management strategies could be adopted to improve collaboration through greater shared understanding in the early stages of design. On the other hand, in the work developed by [21] they contributed by exploring designers' perceptions of shared understanding in remote teams. Analyzing two objectives: to discover the work elements perceived to require shared understanding and, secondly, to identify perceived enablers and barriers to accumulating shared understanding. It was found that team spirit, shared experience, trustworthiness, and transparency, as well as project management and related micro-practices, are perceived as fundamental to generating shared understanding in remote design teams.

    2.2 Measuring shared understanding

    Research has also been conducted on how to measure shared understanding, and some work related to these measures is presented below: In [22] it is suggested that one way to measure shared understanding between agents (whether individuals or groups) is to assess the structural isomorphism of the cultural models developed by the agents in question. If the models are identical, then the level of shared understanding between the agents involved will be at its theoretical maximum. If the models do not resemble each other, the shared understanding will be minimal. Similarly, in Smart [12] the measurement of shared understanding is studied based on the assessment of shared skills. To do this, a cultural model is used in which the nodes of this model represent concepts and associated properties, while the links between concepts reflect the community's beliefs about the relationships and dependencies between concepts. In [23] in a study with interprofessional emergency medical teams, emergency medicine residents, nurses, and medical students independently performed, recorded, and coded resuscitation simulations. This study allowed measurement of the team's perception of shared understanding according to the information provided and a measure of the team leader's effectiveness.

    2.3 Shared understanding in software engineering

    On the other hand, those studies that have been carried out in the field of software engineering are shown, which is considered knowledge-intensive, making it particularly dependent and with a need to achieve shared understanding in its various actions performed. This is why knowledge management to enable collaboration within the software team and between the team and its stakeholders have received much attention [24]. Nakakawa et. al. [25], explore ways to decrease the lack of shared understanding among stakeholders in collaborative enterprise architecture development by adopting situational method engineering to guide the development of a method to enable stakeholders to acquire a shared understanding of requirements for enterprise architecture. On the other hand, Hsieh [10] identifies geographical, temporal, organizational, and cultural boundaries as barriers to shared understanding in distributed requirements engineering, introducing a theoretical framework to investigate the impact of culture. For their part, in the project defined in [26] they developed a framework to investigate the interaction of factors that shape shared understanding and shared commitment during agile distributed information systems development project team interactions, it was found that shared interaction shared understanding, and shared commitment in this type of team are determined by the dynamic interaction between macro-level (contextual) and micro-level (localized) factors. On the other hand, Dossick et. al. in [27] show the first findings of an empirical study that seeks to explore the use of Photo Elicitation techniques in combination with ethnography to assess the amount of shared understanding in multidisciplinary teams working on a building design project. In addition, the construction management, and visualizations that these students created and used to learn and develop integrated skills were studied. The results of two studies are shown in [28]: an experiment with students and a pilot field study with professionals using a content validity survey instrument to measure shared understanding in companies and IT professionals that aims to monitor the relationships between companies and IT units in their organizations.

     

    3. METHODOLOGY

    The research shown in this paper was developed following the multi-cycle action-research methodology with bifurcation [16]. This methodology allows the management and development of research projects based on several research cycles that are presented in detail below.

    3.1 Conceptual cycle

    This cycle consisted of an analysis of the existing literature, in which the main problem within the area of knowledge to be addressed is identified as the first element, determining that one of the main problems of collaborative work is that successful collaboration is difficult to achieve [29]. At the same time, it is stated that group members involved in collaborative activities will need to coordinate with their peers the tasks to be performed to meet the objectives [30] and one way to coordinate this is to achieve a shared understanding among all [31]. Accordingly, a literature review and analysis were conducted, which aimed to identify existing elements that could be included in the definition of the THUNDERS process. In addition to considering the requirements of collaborative problem-solving activities, computer-assisted collaborative work, and heterogeneous groups, which would be our defined context where the process would be used.

    3.1.1 Literature review

    The review aimed to characterize and identify the different existing approaches (process, activities, phases or steps, techniques, and strategies) that would later help to define THUNDERS with existing elements, or the construction of new ones if necessary. The literature review work was addressed through the next research questions:

  • What approaches have been used to execute computer-supported collaborative work?
  • Do the approaches consider the shared understanding construction in their definitions?
  • Do the approaches use any formal measurements to validate shared understanding?

    • The data sources that were used for literature review development are: IEEE Computer Society Digital Library and Scopus. In the search strategy, the keywords were identified with their respective synonyms, and through the combination of these keywords and their association, the search string was developed.

    To make the study selection, a set of inclusion and exclusion criteria that allowed us to verify their quality and guarantee that they were studies related to the computer-supported collaborative work were defined.

    Then, the identification and selection of the primary studies were based on two main steps:

    Step 1: Consisted of applying on each of the data sources the search string; in this way, we obtained for IEEE = 10 papers and for Scopus 263 papers. In this step, we also carried out a debugging process of the recovered studies, which consisted of identifying the studies that were repeated and others that we consider as useless.

    Step 2: In order to reduce the application subjectivity of the inclusion and exclusion criteria, several researchers participated in this step (Two from the Universidad de la Plata and one from the Universidad del Cauca). In the first iteration of this step, the application of the criteria was done by reading the title of the article, the abstract, the keywords and the conclusions. In this process, each researcher read these sections, and at the end, a meeting was held among all participants to reach consensus and determine the inclusion or not of each article reviewed. As a first iteration result, 30 papers were gathered as possible primary studies. In the second iteration, the criteria were applied by reading the full content of those 30 papers. In this iteration, the same review dynamic was maintained by the authors as in the first iteration, and as the result was obtained a set of 12 papers were classified as primary studies.

    From all this process, it was evidenced that in most of the found literature there is no complete approach that ranges from the design of the activity to the complete verification of compliance with it. In addition to not considering shared understanding, as a strategy to improve collaboration, an aspect that will be considered in this work, is what and how to achieve it. With these 12 primary papers, it was possible to identify elements that served as the basis for the creation of the THUNDERS process proposed.

    3.2 Methodological cycle

    According to Bittner and Leimeister [19], it is determined that little is known about what leads to shared understanding, besides that the actors need guidance on how to evoke processes deliberately and repeatedly to achieve it. Considering the above, this work defines a process that can determine how to achieve a shared understanding. In this sense, this cycle consisted of the THUNDERS definition, where initially the components that would be part of the process were identified, starting from the analysis of the information previously obtained in the literature review. After this identification, a definition was made out of all the components that would make up the process, those obtained from the review and those that should be created. This, in order to create a version of THUNDERS, with phases, activities, tasks, steps, and work products that will allow to execute a collaborative work in problem solving activities seeking to achieve a shared understanding.

    For this, this work is based on the concept of software process to determine the outcome of this research, a definition that refers to a sequence of steps arranged with some kind of logic that focuses on achieving some specific results [32]. With this, it is determined that the process obtained will be defined at a conceptual level, which will be an ordered set of methods and activities, tasks, practices, guidelines, strategies, rules, steps, roles, inputs, and outputs.

    In order to define this THUNDERS process, the collaboration engineering design approach is followed [33], which addresses the challenge of designing and deploying collaborative work practices for high value recurring tasks and transferring them to practitioners to execute them by themselves without the ongoing support from a professional expert in collaboration [34]. To model THUNDERS, it is used SPEM 2.0 meta-molding (Software Process Engineering Meta-Model) [14]. The context in which this research will be worked refers to small and heterogeneous groups, in addition to applying the process in problem-solving activities.

    From this point of view, the objective is to address this challenge by providing a structured collaboration process based on theory-grounded-design guidelines that can be used to support heterogeneous groups to develop a shared understanding of a task. With this, it is intended to contribute to making the construction of shared understanding more predictable and manageable.

    3.2.1 Process formalization

    THUNDERS contains a number of elements defined from different aspects. Initially, from the process specification aspects, it contains the following elements:

  • Process: In the context of this research, the definition of process is used which determines that: is a sequence of steps arranged with some kind of logic that focuses on achieving some specific result [32]. In this sense, THUNDERS is a process (defined at two levels: the conceptual level that defines how to execute a collaborative activity through strategies, activities, tasks, rules, steps, roles, inputs, results, and a technological level that provides the technical support to achieve it and allows the process to be easily reusable) that allows the design, execution, and validation of a collaborative activity, which determines a sequence to guide the needs of collaborative work in a systematic way.
  • Process roles: It defines who is in charge and responsible for executing a specific task. Roles are a set of related skills, competencies, and responsibilities of an individual or a group. It may happen that an individual plays several roles or that a role is played by several individuals.
  • Phases: THUNDERS defines 3 phases, which refer to a significant period of the process and ends with a major management control point, milestone, or set of completed results
  • Activities: Each THUNDERS phase contains a set of defined activities, representing an overall unit of work.
  • Tasks and steps: Each THUNDERS activity contains a set of tasks and each task a set of steps that are the basic, atomic building blocks of the process, representing the effort to be performed. Tasks affect some work products and link roles for their execution.
  • Workflows: THUNDERS contains a set of workflows that are the operational aspects of an activity and show the flow of each of them.
  • Work products: THUNDERS tasks consume, produce, or modify work products, which can be of type Artifact: of tangible nature (model, document, code, files...). Deliverable: provides a description and definition to package other work products for delivery. Outcome: of an intangible nature (result or state), or that is not formally defined.
  • Guidelines: A process element that guides in detail the execution of some aspects of THUNDERS.

    • Specifically, according to the THUNDERS process, the computer-supported collaborative work is divided into 3 phases, Pre-Process, Process, and Post-Process, which were taken from Collazos’ work [35], these phases were updated and adapted to collaborative work. The first Pre-Process phase begins with the specific design of the activity and the necessary elements to be carried out; in the Process phase, the collaboration activity is executed in order to achieve the objectives based on the interaction among group members and with necessary resources. In the end of the activity, in the Post-Process phase, the activity leader (the person in charge of guiding the activity) performs an individual and collective review to verify the achievement of the proposed objectives and the problem that was solved in the activity.

    Pre-Process phase: Each activity was assigned its respective description and workflow to achieve its objective. The Table 1 shows the process elements of the Pre-process phase as activities, tasks and roles, The input and output work products, for each task, can be seen in detail in [13].

    Table 1. Process elements of the Pre-Process phase
    Source: Created by the authors.

    Process phase: This phase is where the collaborative work interactions take place and obtains shared understanding through different strategies. The Table 2 shows the process elements of the Process phase.

    Table 2. Information about the Process activities
    Source: Created by the authors.

    Considering the importance of the Shared Understanding activity, the tasks and steps that are part of its definition are detailed below. This activity seeks to have the group members agree on what the problem is in the collaborative activity; they must understand it before starting its development, this activity is formed by the Tacit Pre Understanding task which is, the people's ability to understand individual representations when they make use of them [36]. The Construction task happens when one of the group members inserts meaning by describing the problematic situation and deals with it, hereby tuning in fellow teammates. These fellow teammates are actively listening and trying to grasp the given explanation [37], the Collaborative Construction task is a mutual task of building meaning by refining, building, or modifying the original explanation [38], and finally, the constructive conflict task, which is where the differences of interpretation among the group members are treated through arguments and clarifications [6]. Considering these tasks, it is defined for each a series of steps that will allow to achieve the objectives (see Table 3).

    Table 3. Steps of each task of the shared understanding activity
    Source: Created by the authors.

    Post-Process phase: The phase seeks to verify that the objectives proposed in the activity were achieved and if the problem was solved, in addition to verifying the participants' performance. The following Table 4 shows the process elements of the Post-Process phase.

    Table 4. Information about the Post-Process activities
    Source: Created by the authors.

    For the THUNDERS process, a set of characteristics were also established so that each of the elements of the process would follow the previously defined characteristics, among which are the following:

    Integrated: To collaborate, group members have permanent interactions, exchanging ideas or points of view, for which it is necessary to have adequate communication, which generates that the group understands and agrees on the interpretation of the tasks to be performed, what and how they will do the work together, and understand the subject on which the activity is executed, i.e., it is necessary to build a shared understanding, to obtain as a consequence better levels of collaboration [10]. To obtain all of the above, THUNDERS integrates concepts and elements of collaboration and shared understanding so that from the construction of shared understanding a more assertive communication is generated, and therefore when working in a coordinated manner there is greater collaboration, to integrating specific elements for problem solving activities and defined for heterogeneous grouping of participants.

    Assisted: Little attention has been paid to the systematic development of processes that lead to shared understanding within heterogeneous groups [39] in addition to the lack of knowledge about the specific patterns that lead to their construction, and the non-existence of clear and adequate execution flows [6], which is why practitioners need guidance on how to evoke processes in a deliberate and repeated way [19]. THUNDERS provides elements to execute each of the phases of collaborative work in a guided way, providing a step-by-step that is supported by activities, tasks, work products, guidelines, and roles, in order to design, execute and validate a collaborative activity to successfully build shared understanding. In addition to offering a set of recommendations and assistance documents, which provide already built elements of the collaborative activity, or support according to the needs that arise.

    Materialized: Shared understanding is one of the five critical challenges to be achieved within collaborative activities, critical due to, for example, the lack of overlapping experiences; the context, the shared language of actors; the ambiguous nature of problems; or the disruption of routines, which influences how a group forms and performs [40]. Future research should aim at a better understanding of the complex and still ambiguous phenomenon, its antecedents, and effects, thus generating promising opportunities to further develop techniques that leverage the benefits of shared understanding for effective group work [19]. THUNDERS contains work products, workflows, and mechanisms that enable shared understanding to be built and subsequently measured in a way that makes it tangible and materializable.

    3.3 Evaluation cycle

    This cycle consisted of the evaluation of THUNDERS. This evaluation was divided into three parts, initially, it was validated through experts, who made their observations and corrections to the structure of the process in terms of syntax and semantics, identifying the need to modify several elements, the quality of its SPEM 2. 0 using AVISPA, identifying some errors in the definition and formalization of the model, which was discussed and analyzed to determine the solutions that were incorporated from its design and formalization, after applying these improvements, the feasibility and usefulness of the initial process proposed for the construction of shared understanding were investigated through an experiment.

    3.3.1 Expert validation of the structure

    The objective of the expert validation was to analyze the structure of the version of the process, validating the syntax and semantics of the process, in such a way that some errors, excess or missing elements in the process specification made in SPEM 2.0 [14] were identified.

    Three experts in software and process engineering participated in the validation. The experts were: A Doctor in Electronic Sciences from the Universidad del Cauca, with 10 years of experience in software engineering, design, construction, specification and improvement of processes and process lines. Two systems engineers from Unicomfacauca, who have two years of experience in process design and specification, deepening in SPEM 2.0 modeling.

    To carry out the validation by expert judgment, a set of activities were designed that made it possible to perform all the actions necessary to obtain the expected results. These activities, with their expected duration and the instruments used, are presented in Table 5 below.

    Table 5. Design of expert validation activities
    Source: Created by the authors.

    Expert validation execution: All the activities planned for the expert validation were executed, using the tools provided for their support, using a time of 2 hours and 20 minutes. Several important elements were obtained to improve the specification of the process and give it a better structure, which will be shown in the following section.

    Results obtained: After each of the activities planned for the expert validation were carried out and each expert responded to the established survey, the following results were obtained. The following are the results of the experts in general terms.

  • For each of the phases of the process, it was inquired whether the names and descriptions were adequate, clear and whether the objective of each phase was understood. The results showed that 66.7 % of the participants said yes, and 33.3 % said that they needed to be improved.
  • For each of the activities of the Pre-Process phase, it was asked whether the names, descriptions, and workflows were adequate and clear, and it was determined that 33.3 % of the experts thought they were adequate and clear and 66.7 % thought they should be improved. The same was done for the activities of the Process phase, where it was determined that 66.7 % of the experts thought they were adequate and clear and 33.3 % thought they should be improved. The same was done for the activities of the Post-Process phase, where 66.7 % of the experts thought they were adequate and clear and 33.3 % thought they should be improved.
  • For each of the specific tasks of each activity, and for each phase, a more detailed analysis was made, where the experts were asked if the objective of each task was understood and if the names and descriptions were adequate and clear. If the task had associated input and output work products and if these had an adequate, clear, and understandable description, and if they had associated adequate guidance. Whether the task had primary performer roles, and additional performer roles appropriate to what they were to perform, with clear, understandable role descriptions and associated skills. If the task defined clear and adequate steps that allowed the complete and correct execution of the task. For this inquiry, observations and recommendations for improvement were solicited if necessary for each specific element.

    • Elements to correct or improve the process: Based on the survey conducted, where the experts made their observations and corrections to the structure of the process regarding its syntax and semantics, it was possible to identify the need to modify several elements, among which were the following:

  • The descriptions of all Pre-Process activities are not clear, and it is necessary to specify the need to execute them based on theory.
  • There are workflows that are not clear to follow in the activities.
  • Activities should not have assigned roles; roles are task specific.
  • Some task names are not clear.
  • There are work products that do not correspond as task inputs and outputs.
  • It is necessary to review the roles assigned to the tasks so that support between roles can be generated.
  • There are attendance documents that are not clear to follow.
  • It is necessary to establish a color code for the attendance documents to know what is mandatory to fill out and what is not, and if it is information that is brought from other tasks.
  • It is necessary to define if all the tasks are always mandatory.

    • The most relevant results for each expert are shown individually:

    Expert 1:

  • The Process phase is often confused with the name of the complete method.
  • The workflow for the activity "Define pre-conditions for group members" in the Pre-Process phase is unclear, and its usefulness for the execution of the Process is not apparent.
  • The activity "Design the verification method of problem-solving" workflow does not have a clear thread and should be revised.
  • The "Collaborative activity" attendance documents do not support the clear execution of the activity, not everything is mandatory, and it is not known how to use it.
  • The roles must collaborate with each other so that the execution of the tasks is more adequate.

    • Expert 2:

  • The description of the Post-Process Phase needs to be improved, as it is not understandable.
  • When defining a process, it is a mistake to assign roles to activities; roles are specific to each task.
  • The "Collaborative activity" workflow does not have a proper sequence considering the collaborative problem-solving activities.
  • The objective defined for the activity "Verify compliance with the problem" is not very descriptive; it is necessary to improve it to see its usefulness.

    • Expert 3:

  • The names of the phases are not very meaningful.
  • An activity should not have roles.
  • The assistance documents of the activities of the process should guide the user in a better way because they are not very intuitive.
  • The names of the tasks “Constructive Conflict” and “Construction” are difficult to understand, regarding their purpose and the need to have them within the process.

    • With the results obtained from the expert validation, the analysis of the improvements that should be included in the process was obtained:

  • The syntactic and semantic errors of the process, referring to what is specified by SPEM 2.0, which do not correspond, should be corrected.
  • The assignment of both input and output work products should be corrected to avoid isolated or overloaded products.
  • Tasks and work products (inputs and outputs) should be given more representative names to clarify them.
  • All descriptions of phases, activities, and tasks should be improved, including support and theoretical references to define each description.
  • Workflows of activities and each task should be improved.
  • Roles in each task should be reviewed and redistributed, looking for support among them.
  • Tasks should be defined in each activity, exclusively dedicated to the follow-up of each step, and focused on verifying compliance with the specified requirements in order to be able to make corrections or improvements at the required time.
  • Assistance documents should be reviewed to redefine the wording of the steps to be followed and include a color code to facilitate their completion.
  • For each task of the process, it should be determined which ones should be mandatory and which ones should not. This is necessary when it is required to define a collaborative activity since it is designed, executed and the different elements are validated.

    • 3.3.2 Validation of the THUNDERS specification

    In order to have a more reliable and valid version from the viewpoint of the formalization in SPEM 2.0 of THUNDERS process model, before its experimentation, an initial assessment was executed where AVISPA-Method (Incremental method for visual analysis of process models) was used [15], which allows the assessment of the process models at a lower cost than its assessment in the real application. AVISPA-Method uses a tool for the analysis and visualization for a Software Process Assessment called AVISPA [41] and defines the following set of activities to guide the assessment: a) To design the process model: a process model version (SPEM version) is designed and formalized. b) To export the process model: the process model SPEM version is exported to an XML version. c) To examine the process with AVISPA-Method: the process model (XML version) is loaded in the AVISPA tool. In each of the views generated in AVISPA and with the help of error patterns [15], potential problems, and opportunities for model improvement are identified and located. d) Analysis and results report: the identified problems in the process model are reviewed and discussed. This analysis requires reviewing the process model in its original format (SPEM version). In the end of the review, the real problems on the process model are identified, as well as the possible improvements to be made. e) Adjust: adjustments are made to the process model according to the errors and suggestions for identified improvement. In the following section, an analysis of the results obtained from applying the AVISPA-Method in the THUNDERS process model assessment is shown. The results are detailed from the three graphic views (tasks, roles, and work products) provided by AVISPA, each view deals with a particular aspect of the process model [41].

    Task view of the THUNDERS process model: The task view provided by AVISPA shows the process from the perspective of the tasks performed during the execution of the process model. In this view, each rectangular node represents a specific task of the process and the attributes of each node provide information about the process that is being analyzed [41]. Figure 1 presents the results obtained when assessing THUNDERS regarding the tasks, which shows that several possible errors were found in the specification of the process model. According to the error pattern, independent sub-projects [15] groups of isolated nodes can be identified in the task view. In Figure 1 it can be seen that there are two set of disconnected tasks (subgraphs at the top of the Figure 1), which refers to the fact that these sets of tasks do not add value to the process objective and, therefore they act in isolation and do not help achieve the objective of the process. The first set (task group in blue color) of disconnected tasks is formed by "Determining group’s characteristics, defining prior knowledge, applying characterization mechanisms, and Analyzing the prior knowledge", and the second set (task group in yellow color) by the tasks that "Give feedback on the activity and Close the activity". Furthermore, in this view it can be seen that the tasks with the numbers 1, 2 correspond to evaluate the problem resolution, and evaluate the achievement of objectives, respectively. Taking into account that the width of the nodes represents the outputs produced by the tasks, in the specific case of tasks 1 and 2, their width exceeds the average with respect to the others, which allow to identify the possible existence of the error-pattern-multipurpose task, which refers to the fact that a task must focus on achieving a specific purpose, instead of generating work products that misled the task that is the basic work unit of the process.

    Figure 1. Task view
    Source: Created by the authors.

    Work products view: To verify the process model regarding the work products, AVISPA provides a view for this purpose, allowing to see over-demanded work products [15]. Figure 2 shows that the work products with the numbers 1, 2, and 3, correspond to the List of objectives to be achieved in the activity, Activity problem with its information, and the List of each participant with their respective role. Considering that, the height of the nodes represents that they are over-demanded products, that is, they are inputs to multiple tasks of the process. In this specific case, their height exceeds the average with respect to the others, which made it possible to identify the existence of the error pattern in over-demanded work products. Furthermore, in the upper part of Figure 2, there is an isolated subgraph, formed by the set of work products in green color, Information on the results obtained in the activity and Lessons learned from the participants, this isolated graph allowed to identify the possible existence of the error pattern of independent subprojects.

    Figure 2. View about over-demanded work products
    Source: Created by the authors.

    In Figure 3, the nodes also represent a work product, but this view emphasizes on the nodes that may be useless, showing that the dark blue nodes identify the possible existence of the Waste work products pattern, that refers to work products that are neither deliverable nor input for any task.

    Figure 3. View about unnecessary work products
    Source: Created by the authors.

    Roles’s view. In the view role, each node identifies a role, and each of the lines among nodes specifies collaboration[41]. Figure 4 presents the results obtained when evaluating THUNDERS concerning its roles, nodes are obtained in the form of squares and rectangles, and each of them refers to a role, where it can be observed that none of the nodes is interconnected with others, i.e., no role collaborates with the others, which allows recognizing the presence of the error pattern Isolated Role [15], since as far as possible it is necessary to define roles that collaborate with each other [42].

    Figure 4. Roles’s view
    Source: Created by the authors.

    Based on the results obtained in the validation of the version (by experts and AVISPA), some changes, corrections, and improvements were made, thus obtaining a new version of the process. These new elements were:

  • Corrected the English syntax in the name of the activities and tasks.
  • A new variability is assigned for each task, and for those tasks that are optional, the contexts and situations when they should be executed are defined.
  • Task names are changed to make them clearer.
  • Some tasks are repositioned to be executed as inputs to other tasks.
  • Some assistance documents are improved in order to make them easier to use.
  • Some task descriptions are better written to make them clearer.
  • Task inputs and outputs are improved to improve the relationship between them.
  • Relationships between process roles are analyzed and corrected.

    • 3.3.3 Validation of the THUNDERS viability and usefulness

    To validate the THUNDERS viability and usefulness, an experiment that is presented in summary in the following section was carried out.

    Experiment context: The experiment was conducted in a university environment in which 45 last-semester students of Universidad de la Matanza - UM (Argentina) participated with a well level of experience in the activity topic, for this group the proposed process was applied. Moreover, 15 students of Universidad Nacional de la Plata - UP (Argentina), enrolled in the last year, participated with a well level of experience in the topic, to whom the proposed process was not applied. The groups were formed using a software tool called Collab [43] that analyzes the learning styles and organizes the group through a Genetic algorithm described in [44], where heterogeneous groups of 5 participants were formed and allowed learning styles to complement each other and thus obtain better results.

    The problem-solving activity consisted of each group assuming that they were part of the engineering team process of a company, where they had to establish the software development processes that best adapted and supported the projects in the company. To solve the problem, they had to follow an execution guide called SpeTion-SPrl, where information about the projects and processes is defined, and with this, the scope could be determined.

    Experiment planning: The research question was defined as: How feasible and useful is this proposed process? This study had one analysis unity, which was the academic context, where a problem-solving activity about the Scope definition in Software Process Line was carried out.

    Hypothesis. Considering the objective, it is intended to evaluate the following hypotheses:

  • The proposed initial process is feasible for the construction of shared understanding in a problem-solving activity
  • The proposed initial process is useful to achieve the objectives of the problem-solving activity

    • Execution of the experiment: The UM groups applied the entire process, while the UP groups simply met to develop the proposed activity. Therefore, the UM in the Pre-Process phase for each activity used a software tool MEPAC [45], which provided the step by step through forms, with the design and definition of necessary elements. The Process phase used a software tool Collab [43] for group formation; also in this phase formats were used to write the individual’s understanding about the problem, to write the questions or disagreements, to classify the understanding of the other members, to classify their own understanding, the group also wrote the understanding where everyone agreed the groups solved the problem and used a survey format with 24 questions to analyze the results (Each of the instruments and their formats can be consulted in the Supplementary Material).

    The time used to apply the proposed process in UM was 3 hours 55 minutes, and for the UP it was 2 hours and 40 minutes.

    3.3.4 Analysis of experiment results

    With the observation made by the researchers while the activity was being carried out, it was possible to determine that those groups that obtained poor results in the evaluations were those that did not perform well in the application of the process and did not generate internal discussions to resolve doubts. Therefore, it was observed that following the process was exhausting for the participants and that this generated a lack of commitment to the rest of the activity and a high cognitive load. On the other hand, to guarantee that the found results are not only observational and apparent but statistically significant, the student's t-distribution was used [46], which allowed to validate the specific hypotheses (the details of the results obtained in the validation can be seen in [47]).

    The specific hypotheses to validate that the initial process was feasible are:

  • H.1.1. Improvement in the participants’ descriptions about what they should do.
  • H.1.2. The participants understand and agree on the descriptions from their other groupmates of what should be done.
  • H.1.3. Improvement in the homogeneous understanding and the discrepancy between each participant with others, about what they should do.
  • H.1.4. Improvement in the activity results of the Shared Understanding stage.

    • Similarly, to validate the usefulness of the process, the following specific hypotheses were defined:

  • H.2.1. Improves in the quality of the final obtained results when performing the problem-solving activity.
  • H.2.2. The number of posed questions to the activity coordinator decreases.
  • H.2.3. Improves the perception of the participants' satisfaction, about the achievement of the activity objectives.
  • H.2.4. The use of the process improves the perception of the participants' satisfaction with the process elements and with the activity outcome.

    • Considering these specific hypotheses, applying the mechanisms, and performing the statistical analysis, the following results were obtained for each specific hypothesis (See Table 6).

    Table 6. Results for each specific hypothesis
    Source: Created by the authors.

    According to the validation of hypotheses H.1.1, H.1.2, H.1.3, and H.1.4 related to the feasibility of the process, it can be said that the process is feasible to build shared understanding in a problem-solving activity. According to the validation of H.2.1, H.2.2, H.2.3, and H.2.4, it can be concluded that the process is partially useful in achieving the objectives of the problem-solving activity, but it cannot be assured that the process improves the participants' perception of satisfaction with its elements and with the outcomes of the activity. Considering that the process is perceived as feasible and partially useful, it can be inferred that although good results are obtained, a high cognitive load needs to be improved.

    With the statistical comparison of the results with the use of the process and without its use, it was verified that the THUNDERS process improves the participants' individual understanding enhances the group’s understanding and generates a homogeneous understanding of the activity, it does not generate a discrepancy of each participant regarding the group understanding, the shared understanding activities generated better results and were better fulfilled among the participants. Also, it was determined that the use of THUNDERS process generates final products with better quality levels. THUNDERS allowed to obtain better achievement participants' satisfaction with the objectives proposed by the activity. Conversely, it cannot be determined that the THUNDERS elements are satisfactory for the participants and in the same way, with the outcomes of the activity.

     

    4. CONCLUSIONS AND FUTURE WORK

    This This paper, being an extension of the article presented in [13], presents in more detail the validations performed to the first version of the THUNDERS process, which is a process that guides and defines the step by step for the construction of shared understanding in problem solving activities, a process that was built from elements found in the literature review, and the analysis of the context and the identified needs.

    Here it is shown how THUNDERS was validated in three parts, in the first one, its formal specification was validated through experts to determine the correctness and completeness of its structure, where errors were identified in the definition of THUNDERS, errors that were corrected to generate a new version, finally determining by the observations given that the process was correct and complete, however it needed to correct some of its elements, after that the quality of its SPEM 2 specification was validated. 0 using AVISPA, this validation allowed to identify some error patterns in the definition and formalization of the model, which were discussed and analyzed to determine solutions that were incorporated from its design and formalization. This validation allowed direct efforts to improve the definition of THUNDERS and thus have a more stable and validated version, from the point of view of its formalization. In the third part of the validation, an experiment was conducted to investigate the feasibility and usefulness of the proposed process for the construction of shared understanding. The results obtained in the experiment from the statistical analysis allowed us to conclude that THUNDERS is a viable process for the construction of shared understanding and useful for achieving its objectives. However, according to the specific null hypotheses that were accepted, it cannot be determined that it improves the participants' perception of satisfaction with the process's elements and the activity's results. In addition, the need to improve the process so that it is lighter and easier to perform in order to avoid cognitive load at the time of its use and application due to the number of elements that need to be defined and the amount of time required to do so, was identified.

    As future work, it was identified that it is still necessary to improve elements of the definition of the process, both in its definition, structure, and conceptual part, in this sense, although existing measurement elements were used for shared understanding, it is still necessary to include in THUNDERS monitoring and assistance mechanisms that allow maintaining its construction throughout the activity, because in the way of interaction it can be lost or can become inadequate understanding. In the same way, it is necessary to incorporate in THUNDERS mechanisms that allow to achieve a better shared understanding and a greater ease of use, developing some techniques and elements to take advantage of its advantages. Elements such as more specific templates and guidelines to support the creation of an activity to build shared understanding, specific roles, and additional elements that can be incorporated into the process of building shared understanding. Similarly, more experimentation is needed to determine the elements that make building shared understanding easier to achieve and faster in the collaboration process.

     

    5. ACKNOWLEDGMENTS AND FUNDING

    This work did not receive financial support from any entity.

    CONFLICTS OF INTEREST

    None declared.

    AUTHOR CONTRIBUTIONS

    Vanessa Agredo-Delgado has contributed to the design, execution and analysis of the results obtained in each of the validations performed as well as in the writing of the article.

    Pablo H. Ruiz has contributed with the design of the validations and with corrections in the writing of the results obtained.

    Cesar A. Collazos has contributed with the definition and execution of the methodology used for the specification of the process elements.

     

    7. REFERENCES