Commit graph

29 commits

Author SHA1 Message Date
He Yin Ling
45a50ed701 test: fix incorrect error type in xunit result 2019-12-11 15:57:49 +11:00
He Yin Ling
6e05a79536 test: update example and unit tests with new import roles:
tiny_test_fw is a python package now. import it using normal way.
2019-12-11 15:57:49 +11:00
He Yin Ling
69c0e6243e unit-test-app: fix test script error when handling multiple unit test configs in one config file:
If we have multiple configs, we need to flash DUT with different binaries. But if we don't close DUT before apply new config, the old DUT will be reused, so new config name will not be applied.
2019-12-11 15:57:49 +11:00
He Yin Ling
1041bc800c unit-test-app: print config name in test script 2019-12-11 15:57:49 +11:00
Ivan Grokhotkov
7db756fbd9 tiny-test-fw: add build config and target options 2019-12-11 15:57:49 +11:00
He Yin Ling
304e7119ce ci: fix random unit test CI failure:
we use `-` command to check if DUT reset pass. If we input `-` command
during DUT bootup, DUT could only receive `\n` and print test cases.
Print test cases could take long time and lead to reset check timeout.
Now we will add delay after reset, and enlarge reset check timeout to
solve this problem.
2019-06-18 12:22:00 +08:00
He Yin Ling
b75400294a test: collect unit-test performance items and save to report 2019-04-19 14:39:44 +08:00
He Yin Ling
0a3975e80d test: handle exception by unit test script:
unit test script will detect exception, we don't need to let test fail
when exception happened.
2019-03-19 11:24:08 +08:00
He Yin Ling
323a790f01 test: fix CI UT job pass even ut app bootup fail 2019-03-10 06:21:37 +00:00
He Yin Ling
431653bba8 unit-test: fix 2 issues in unit test script:
1. Gitlab will only regard failure (not error) as failed case. Use log
failure instead of log error, to display all error msg in test result
2. fix failed case not recognized by test script when failed during
reset DUT
2019-01-10 09:51:17 +00:00
He Yin Ling
783cb692e8 unit-test-app: support passing parameter with signals 2019-01-10 09:51:17 +00:00
Roland Dobai
bfa9610f58 tools: Fix the Python coding style 2018-12-19 11:56:24 +01:00
Angus Gratton
0a27cfa850 unit_test.py: Allow importing tiny-test-fw from IDF_PATH as well as TEST_FW_PATH 2018-12-18 14:09:05 +11:00
He Yin Ling
b0154cbf58 unit-test-app: fix the RegEx to detect test case:
tags are not correctly parsed if eol do not contain `\r`
2018-11-29 23:06:16 +08:00
He Yin Ling
5b3d09d5c8 test: fix unit test script code style warnings 2018-11-26 10:07:41 +08:00
He Yin Ling
644571f78b test: generate junit test report according to executed cases 2018-11-26 10:07:41 +08:00
He Yin Ling
2b4e34171b test: fix UT script can't process cmd after reset:
There's still a small chance that can't process command issue still
happen on some runners. Now we change the way to detect if the runner
reset finished: input `-` to check test history for a few times until it
succeeded. If DUT can process check history command, then it should be
able to process other test commands.
2018-10-26 08:42:22 +00:00
He Yin Ling
44e8ec584f test: ut can't process test command:
check ut reset done condition is not correct. If bootup pattern already
exist in cache, we will return without waiting for DUT bootup.
2018-09-19 09:40:59 +08:00
houchenyao
73a763fecb fix the bug that multi device's dut can't detect case 2018-09-12 07:18:00 +00:00
houchenyao
df83323ee3 bugfix:fix the multi device stop thread bug 2018-08-23 08:51:16 +00:00
Ivan Grokhotkov
b600a908c4 unit test: fix regex for detecting abort 2018-08-20 16:49:20 +08:00
Roland Dobai
40486c8723 tools: Allow running unit tests from command line
Add argument parsing to unit_test.py to allow running unit tests from
command line.
2018-07-30 13:05:41 +02:00
He Yin Ling
bffcea09df CI: erase nvs partition before test:
Latest NVS partition bin can't be parsed by old IDF revision. Need to erase before test.
2018-07-27 17:22:24 +08:00
houchenyao
9223639010 stop main thread when one failed 2018-06-26 11:02:23 +08:00
He Yin Ling
3b24bc42d2 CI: assign unit test cases according to sdkconfig:
We have built unit-test-app with different configs.
Currently we use the config name as tags to match runners.
It's not easy to add new configs (need to update tags to existed runners).

Now we'll parse required test runner tags from `sdkconfig` file.
For example, if config enables `CONFIG_SPIRAM_SUPPORT`, then it requires `psram` tag.
This will make adding new configs easier.

In this commit we change the one behavior of assign test: match keys of
cases should be exactly the same with job tags. This fixes cases select
jobs include their tags, and jobs requires those tags can't be assigned.
2018-05-24 03:40:03 +00:00
Angus Gratton
e8525396dd test: Allow per-unit-test timeouts
Work around for "(WL) write/read speed test" taking >30s in some configs
2018-05-21 12:27:10 +00:00
Ivan Grokhotkov
48379b8bbe unit-test.py: increase timeout for test app start up 2018-04-26 18:52:45 +08:00
He Yin Ling
fee844d22c unit-test-app: support multi stages test 2018-03-30 11:38:49 +08:00
houchenyao
5b8a9478a3 CI: new CI ut framework, and can run it in local PC 2018-02-05 06:58:22 +00:00