- 
    
Epic
 - 
    Resolution: Fixed
 - 
    
P2: Important
 - 
    None
 - 
    None
 - 
    None
 
- 
        Module Instruction Abstraction
 
History
Since the beginning of Coin project, it was decided to move build and test instructions under CI source repository to speedup initial development. As the time passed, this has turned out to be expensive long-term solution that increased the CI maintenance burden. Not only because the module instructions would require CI service restart every time that the module instruction is changed but also the new instructions are not tested through a normal CI cycle.
Solution
Since commit ab9875436699c18f1c9133569d3d34032e47e762, Coin yaml -like configuration files that can express Build/Test Instructions that contain commands to be executed on the VM.
How
The functionality is enabled by adding a file module_config.yaml in coin/ sub-directory in the module repository. This has already been done for the CI self tests, which looks like this (old vs. new):
qtqa/tqtc-coin-ci.git:src/modules_ci.py (legacy):
class QtCiModule(Module): """Self tests for the Qt CI.""" async def priority(self, repositoryState: storagestructs.RepositoryState) -> ModulePriority: return ModulePriority.ModuleMatch if repositoryState.project.endswith('coin-ci') else ModulePriority.NoMatch @staticmethod def setup_ssh(instructions: CommonInstructions, moduleConfig: messages.ModuleConfiguration) -> CommonInstructions: """ The ssh config is required for cloning private repositories from Gerrit. """ if Feature.FAKESSHSETUP in moduleConfig.workItemConfiguration.features: instructions.writeToFile("fakefile", "fake data") return instructions path_to_secrets = os.path.join(os.path.dirname(os.path.realpath(__file__)), "../secrets") files_to_write = (("cibot_id_rsa", 0o0600), ("cibot_id_rsa.pub", 0o0600), ("config", 0o0644), ("known_hosts", 0o0644)) instructions.mkdir("/home/qt/.ssh") for filename, mode in files_to_write: filepath = os.path.join(path_to_secrets, filename) with open(filepath, "r") as file: instructions.writeToFile("/home/qt/.ssh/%s" % filename, file.read(), fileMode=mode, hiddenContent=True) return instructions async def build_instructions(self, platformInstructions: CommonInstructions, moduleConfig: messages.ModuleConfiguration) -> messages.AgentInstructions: repositoryState = moduleConfig.repositoryState platformInstructions.installSourceArchive(repositoryState, windowsLineFeeds=False) platformInstructions.cd('{{.SourceDir}}') platformInstructions.make(['-t', 'submodules']) # FIXME Building thrift.protocol might throw warning # PyErr_Format(PyExc_OverflowError, "size exceeded specified limit: %d", limit); # that causes make citest fail unintentionally. platformInstructions.executeCommand(["make"], onIntegrationFailureMessage % "Could not run make.", maxTime=60 * 60, ignoreExitCode=True) platformInstructions.make(["citest"]) return platformInstructions.instructions() async def test_instructions(self, platformInstructions: CommonInstructions, moduleConfig: messages.ModuleConfiguration) -> messages.AgentInstructions: return platformInstructions.instructions()
Note: ssh_setup() method is a hack that can be ignored by the module configuration.
qtqa/tqtc-coin-ci.git:coin/module_config.yaml:
(tqtc-coin-ci) 12:15:32~/tqtc-coin-ci (master) $ cat coin/module_config.yaml 
version: 1
accept_configuration:
  condition: and
  conditions:
      - condition: property  # No cross compilation
        property: host.os
        equals_property: target.os
      - condition: property  # Coin works only on linux
        property: host.os
        equals_value: Linux
build_instructions:
  # We do not have real checkout so we need to convince make that we have up to date submodules
  - type: ExecuteCommand
    command: make --touch submodules
    userMessageOnFailure: >
      Could not update the timestamps on sources. It is probably a bug in the Coin build system.
      Please check if there is no changes to it.
  - type: ExecuteCommand
    maxTimeInSeconds: 3600
    maxTimeBetweenOutput: 60
    command: make
    userMessageOnFailure: >
      Could not build Coin project. Please investigate changes that could cause the failure.
  - type: ExecuteCommand
    command: make citest
    maxTimeInSeconds: 36000
    maxTimeBetweenOutput: 600
    userMessageOnFailure: >
      One or more tests failed. Please check logs to see which ones. There is a big likelihood
      that it is a regression caused by a tested change.
Next
A Jira task will be created for every custom module instructions and linked to this epic
src/modules_3dstudio.py src/modules_automation.py src/modules_ci.py src/modules_custom.py src/modules_ifw.py src/modules_pyside.py src/modules_qt5.py src/modules_saferenderer.py src/modules_yocto.py
Other
Bugs related to the instructions or features that are needed can also be linked to this epic.