[Buildroot] [PATCH] .gitlab-ci.yml: do runtime tests only on explicit trigger

[Arnout Vandecappelle]

On 10/22/18 3:35 AM, Ricardo Martincoski wrote:
> Hello,
> 
> On Sun, Oct 21, 2018 at 08:23 AM, Arnout Vandecappelle wrote:
> 
> [snip]
>>>>  .runtime_test: &runtime_test
>>>> +    # Running the runtime tests for every push is too much, so limit to
>>>> +    # explicit triggers through the API.
>>>> +    only:
>>>> +        - triggers
>>>> +        - tags
>>>
>>> We could also allow a regexp on the branch name to trigger it:
>>>          - /^.*-runtime_test$/
>>>
>>> Using the same commit, I pushed 2 branches:
>>> branch name: gitlabci-triggers, #jobs = 4
>>> https://gitlab.com/RicardoMartincoski/buildroot/pipelines/33723520
>>> branch name: gitlabci-triggers-runtime_test, #jobs = 72
>>> https://gitlab.com/RicardoMartincoski/buildroot/pipelines/33723541
>>
>>  Looks nice, but that's really material for a separate patch (and probably
> 
> Indeed.
> 
>> should be done for defconfigs too then).
> 
> OK.
> 
>>
>>  Would it be possible to also define branches for a specific test or defconfig?
> 
> Yes.
> 
> Using the same commit, I pushed 3 branches and a tag:
> git push gitlab HEAD:test2               #jobs = 4
> https://gitlab.com/RicardoMartincoski/buildroot/pipelines/33765315
> git push gitlab HEAD:test2-TestSyslogNg  #jobs = 5
> https://gitlab.com/RicardoMartincoski/buildroot/pipelines/33765342
> git push gitlab HEAD:test2-runtime_test  #jobs = 72
> https://gitlab.com/RicardoMartincoski/buildroot/pipelines/33765317
> git push gitlab test2-tag                #jobs = 260
> https://gitlab.com/RicardoMartincoski/buildroot/pipelines/33765321

 Nice! I suppose test2-tag is actually a tag, not a branch, so it matches the
'tags' trigger?

> 
> We will need some extra 'sed' trickery to create a regexp for each test case:
> -tests.package.test_syslog_ng.TestSyslogNg: *runtime_test
> +tests.package.test_syslog_ng.TestSyslogNg:
> +    <<: *runtime_test
> +    only:
> +        - triggers
> +        - tags
> +        - /^.*-runtime_test$/
> +        - /^.*-TestSyslogNg$/

 Maybe you can use variables in there? So, in the only specification of
.runtime_test:

    only:
        - triggers
        - tags
        - /^.*-runtime_test$/
        - /^.*-${CI_JOB_NAME}$/

 Hm, unlikely, otherwise the other $ would have to be quoted.

 Apparently nowadays there is also support for more complex only specifications
using variables. Perhaps you could take a look at that? I'm afraid though that
it's always an AND of the conditions you specify, so no way to specify that it's
for tags OR for things matching the CI_JOB_NAME.

 Regards,
 Arnout

> 
> Unfortunately the inheritance using the anchor runtime_test does not allow to
> merge two 'only' keywords, the 'only' declared on the test case key overrides
> entirely the 'only' declared on the hidden key '.runtime_test' (or the other way
> around if << is placed at the end), so we would need to repeat those lines for
> every test case listed.
> But maybe we don't care that .gitlab-ci.yml grows because those lines would be
> automatically generated by 'make .gitlab-ci.yml'.
> 
>>
>>  Could you take care of this?
> 
> Sure.
> 
> Regards,
> Ricardo
>