Commit graph

29 commits

Author SHA1 Message Date
He Yin Ling
4bb656c36d tiny-test-fw: support translate backtrace in IDFDUT 2019-03-19 22:15:33 +08:00
He Yin Ling
3f0c737969 tiny-test-fw: support detect exception in IDFDUT 2019-03-19 14:21:36 +08:00
Ivan Grokhotkov
7ca260554f tiny-test-fw: invoke expect callbacks in order of appearance
Previously the callbacks were invoked in the same order as the list of
patterns to be matched.
2018-12-28 09:56:15 +08:00
He Yin Ling
c229de0564 ci: assign unit test cases in sequence of config and name 2018-11-26 17:22:04 +08:00
He Yin Ling
f298897fb5 test: generate junit test report according to executed cases 2018-11-26 17:10:39 +08:00
He Yin Ling
ab2062aa45 tiny-test-fw: support capture raw data in DUT:
test cases might want to use `expect` and get raw data from DUT at the same time. New added capture method provides a way to do that.
2018-11-26 17:08:27 +08:00
He Yin Ling
3892fd2c09 tiny-test-fw: enhanced junit test report support:
1. replace xunitgen with junit_xml, which can log more info
2. allow test cases to handle junit test report by them own
3. allow test cases to log some info into report via `sysout` tag
2018-11-26 17:08:27 +08:00
He Yin Ling
97610cfccf CI: only create required jobs when use bot to trigger test:
Gitlab CI now allow us to define if a job need to be created by
varialbes. This commit add label to CI jobs, so we can make some jobs
not created in certain scenarios when we trigger pipeline with @bot.
2018-10-26 19:45:24 +08:00
He Yin Ling
1884151ca8 CI: support test one case multiple times by @bot 2018-07-28 15:06:18 +08:00
He Yin Ling
2d31597fa9 tiny-test-fw: support ignore test cases in CI assign test stage 2018-07-28 15:04:14 +08:00
He Yin Ling
715dfac07e tiny-test-fw: fix exception during print debug info:
DUT log is unicode, might not be able to encode
2018-07-28 15:03:58 +08:00
He Yin Ling
bf72ed92a9 CI: erase nvs partition before test:
Latest NVS partition bin can't be parsed by old IDF revision. Need to erase before test.
2018-07-28 15:03:30 +08:00
He Yin Ling
fa9c836f02 tiny-test-fw: save DUT log in different thread:
We found some SD card on Raspberry Pi could have very bad performance.
It could take seconds to save small amount of data.
If the DUT receives data and save it as log, then it stops receiving data until log is saved.
This could lead to expect timeout. As an workaround to this issue,
``BaseDUT`` class will create a thread to save logs.
Then data will be passed to ``expect`` as soon as received.
2018-07-28 15:00:41 +08:00
He Yin Ling
609e39a620 tiny-test-fw: fix incorrect wait timeout in expect 2018-07-28 15:00:28 +08:00
He Yin Ling
3956fe30c0 tiny-test-fw: filter examples with test level 2018-07-28 15:00:16 +08:00
He Yin Ling
831e8d653f tiny-test-fw: print expect failure when test fails 2018-07-28 15:00:01 +08:00
Sagar Bijwe
fd7b794e81 FATFS support without wear levelling
This change allows readonly FATFS to be mounted without wear levelling
support. This will provide the customers a simple way to mount FATFS images
generated on host and flashed onto the chip during factory provisioning.
Since NVS encryption is not supported yet and NVS entry size is limited,
the change will provide an easy alternative for securing the provisioning data
by just marking FATFS parition as encrypted.
2018-06-16 16:02:35 +08:00
He Yin Ling
a68b93d6f3 tiny-test-fw: show descriptive error when test case assignment fails:
Will give hint when any job failed to assign:

```
Please add the following jobs to .gitlab-ci.yml with specific tags:
* Add job with: psram, ESP32_IDF, UT_T1_1
* Add job with: psram, ESP32_IDF, UT_T1_1
```
2018-05-24 03:40:03 +00:00
He Yin Ling
3b24bc42d2 CI: assign unit test cases according to sdkconfig:
We have built unit-test-app with different configs.
Currently we use the config name as tags to match runners.
It's not easy to add new configs (need to update tags to existed runners).

Now we'll parse required test runner tags from `sdkconfig` file.
For example, if config enables `CONFIG_SPIRAM_SUPPORT`, then it requires `psram` tag.
This will make adding new configs easier.

In this commit we change the one behavior of assign test: match keys of
cases should be exactly the same with job tags. This fixes cases select
jobs include their tags, and jobs requires those tags can't be assigned.
2018-05-24 03:40:03 +00:00
Angus Gratton
e8525396dd test: Allow per-unit-test timeouts
Work around for "(WL) write/read speed test" taking >30s in some configs
2018-05-21 12:27:10 +00:00
He Yin Ling
9e7a69be88 example test: integrate wifi throughput test to CI 2018-05-15 13:42:27 +08:00
He Yin Ling
2b068f3ceb tiny-test-fw: fix misc bugs:
1. configs are not functional
    * decorator will be executed when search case, need to set default config before search case.
2. fix DUT encode/decode errors
    * python3 serial don't support write string/unicode, need to convert to bytes first.
    * python2 string could failed to encode/decode non-acsii chars
3. fix bug that log folder not created
4. fix bug that test suite name is not correctly passed:
    * the keyward arg should be `test_suite_name` not `test_name`
5. fix bug that test stopped on failed case
6. fix DUT `read` don't return all data cache
    * `_DataCache.get_data` should first copy all data from queue to data cache and then return to user.
7. fix bug that `expect_all` failed even all expected item passed
8. optimize error info for expect
    * print pattern for regular expression when match failed
9. fix bug that set default config doesn't work
2018-05-15 03:29:31 +00:00
He Yin Ling
98d1f05ab5 tiny-test-fw: add utilities:
1. Attenuator: control programmable attenuator
2. PowerControl: control APC PDU to power on/off devices
3. LineChart: use matplotlib to draw line chart
2018-05-15 03:29:31 +00:00
He Yin Ling
3b3a915552 tiny-test-fw: implement env.get_pc_nic_info:
this method is used to get mac/ipv4/ipv6 address for one NIC
2018-05-15 03:29:31 +00:00
He Yin Ling
fee844d22c unit-test-app: support multi stages test 2018-03-30 11:38:49 +08:00
houchenyao
5b8a9478a3 CI: new CI ut framework, and can run it in local PC 2018-02-05 06:58:22 +00:00
houchenyao
1e4508bc8d tiny-test-fw: fix dut for python2 and python3 2018-02-05 06:58:22 +00:00
He Yin Ling
9395a702ef test: check and collect performance via CI 2017-11-16 17:48:30 +08:00
He Yin Ling
47a9a4a614 test: add test fw for example test 2017-10-31 17:42:19 +08:00