Crash_Override/OVMS3-idf
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Merge branch 'test/UT_master_slave_mode' into 'master'

Browse source 
unit-test-app: support multiple devices test cases

See merge request !1220
This commit is contained in:
Ivan Grokhotkov 2017-12-11 15:22:51 +08:00
parent a4ef4cc376 7a31b93fa3
commit 82aeb18c3f
5 changed files with 299 additions and 29 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

93
docs/api-guides/unit-tests.rst
View file

@ -3,8 +3,8 @@ Unit Testing in ESP32
ESP-IDF comes with a unit test app based on Unity - unit test framework. Unit tests are integrated in the ESP-IDF repository and are placed in ``test`` subdirectory of each component respectively.
Adding unit tests
-----------------
Add normal test cases
---------------------
Unit tests are added in the ``test`` subdirectory of the respective component.
Tests are added in C files, a single C file can include multiple test cases.
@ -31,6 +31,59 @@ Each `test` subdirectory needs to include component.mk file with at least the fo
See http://www.throwtheswitch.org/unity for more information about writing tests in Unity.
Add multiple devices test cases
-------------------------------
The normal test cases will be executed on one DUT (Device Under Test). Components need to communicate with each other (like GPIO, SPI ...) can't be tested with normal test cases.
Multiple devices test cases support writing and running test with multiple DUTs.
Here's an example of multiple devices test case::
void gpio_master_test()
{
gpio_config_t slave_config = {
.pin_bit_mask = 1 << MASTER_GPIO_PIN,
.mode = GPIO_MODE_INPUT,
};
gpio_config(&slave_config);
unity_wait_for_signal("output high level");
TEST_ASSERT(gpio_get_level(MASTER_GPIO_PIN) == 1);
}
void gpio_slave_test()
{
gpio_config_t master_config = {
.pin_bit_mask = 1 << SLAVE_GPIO_PIN,
.mode = GPIO_MODE_OUTPUT,
};
gpio_config(&master_config);
gpio_set_level(SLAVE_GPIO_PIN, 1);
unity_send_signal("output high level");
}
TEST_CASE_MULTIPLE_DEVICES("gpio multiple devices test example", "[driver]", gpio_master_test, gpio_slave_test);
The macro ``TEST_CASE_MULTIPLE_DEVICES`` is used to declare multiple devices test cases.
First argument is test case name, second argument is test case description.
From the third argument, upto 5 test functions can be defined, each function will be the entry point of tests running on each DUT.
Running test cases from different DUTs could require synchronizing between DUTs. We provide ``unity_wait_for_signal`` and ``unity_send_signal`` to support synchronizing with UART.
As the secnario in the above example, slave should get GPIO level after master set level. DUT UART console will prompt and requires user interaction:
DUT1 (master) console::
Waiting for signal: [output high level]!
Please press "Enter" key to once any board send this signal.
DUT2 (slave) console::
Send signal: [output high level]!
Once the signal is set from DUT2, you need to press "Enter" on DUT1, then DUT1 unblocks from ``unity_wait_for_signal`` and starts to change GPIO level.
Building unit test app
----------------------
@ -55,7 +108,32 @@ Running unit tests
After flashing reset the ESP32 and it will boot the unit test app.
Unit test app prints a test menu with all available tests.
When unit test app is idle, press "Enter" will make it print test menu with all available tests::
Here's the test menu, pick your combo:
(1) "esp_ota_begin() verifies arguments" [ota]
(2) "esp_ota_get_next_update_partition logic" [ota]
(3) "Verify bootloader image in flash" [bootloader_support]
(4) "Verify unit test app image" [bootloader_support]
(5) "can use new and delete" [cxx]
(6) "can call virtual functions" [cxx]
(7) "can use static initializers for non-POD types" [cxx]
(8) "can use std::vector" [cxx]
(9) "static initialization guards work as expected" [cxx]
(10) "global initializers run in the correct order" [cxx]
(11) "before scheduler has started, static initializers work correctly" [cxx]
(12) "adc2 work with wifi" [adc]
(13) "gpio master/slave test example" [ignore][misc][test_env=UT_T2_1]
(1) "gpio_master_test"
(2) "gpio_slave_test"
(14) "SPI Master clockdiv calculation routines" [spi]
(15) "SPI Master test" [spi][ignore]
(16) "SPI Master test, interaction of multiple devs" [spi][ignore]
(17) "SPI Master no response when switch from host1 (HSPI) to host2 (VSPI)" [spi]
(18) "SPI Master DMA test, TX and RX in different regions" [spi]
(19) "SPI Master DMA test: length, start, not aligned" [spi]
Normal case will print the case name and description. Master slave cases will also print the sub-menu (the registered test function names).
Test cases can be run by inputting one of the following:
@ -66,3 +144,12 @@ Test cases can be run by inputting one of the following:
- Module name in square brackets to run all test cases for a specific module
- An asterisk to run all test cases
After you select multiple devices test case, it will print sub menu::
Running gpio master/slave test example...
gpio master/slave test example
(1) "gpio_master_test"
(2) "gpio_slave_test"
You need to input number to select the test running on the DUT.

40
tools/unit-test-app/components/unity/include/test_utils.h
View file

@ -36,8 +36,48 @@ void ref_clock_init();
*/
void ref_clock_deinit();
/**
* @brief Get reference clock timestamp
* @return number of microseconds since the reference clock was initialized
*/
uint64_t ref_clock_get();
/**
* @brief wait for signals.
*
* for multiple devices test cases, DUT might need to wait for other DUTs before continue testing.
* As all DUTs are independent, need user (or test script) interaction to make test synchronized.
*
* Here we provide signal functions for this.
* For example, we're testing GPIO, DUT1 has one pin connect to with DUT2.
* DUT2 will output high level and then DUT1 will read input.
* DUT1 should call `unity_wait_for_signal("output high level");` before it reads input.
* DUT2 should call `unity_send_signal("output high level");` after it finished setting output high level.
* According to the console logs:
*
* DUT1 console:
*
* ```
* Waiting for signal: [output high level]!
* Please press "Enter" key to once any board send this signal.
* ```
*
* DUT2 console:
*
* ```
* Send signal: [output high level]!
* ```
*
* Then we press Enter key on DUT1's console, DUT1 starts to read input and then test success.
*
* @param signal_name signal name which DUT expected to wait before proceed testing
*/
void unity_wait_for_signal(const char* signal_name);
/**
* @brief DUT send signal.
*
* @param signal_name signal name which DUT send once it finished preparing.
*/
void unity_send_signal(const char* signal_name);

103
tools/unit-test-app/components/unity/include/unity_config.h
View file

@ -20,21 +20,50 @@
#define UNITY_OUTPUT_FLUSH unity_flush
// Define helpers to register test cases from multiple files
#define UNITY_EXPAND2(a, b) a ## b
#define UNITY_EXPAND(a, b) UNITY_EXPAND2(a, b)
#define UNITY_TEST_UID(what) UNITY_EXPAND(what, __LINE__)
#define UNITY_TEST_REG_HELPER reg_helper ## UNITY_TEST_UID
#define UNITY_TEST_DESC_UID desc ## UNITY_TEST_UID
// get count of __VA_ARGS__
#define PP_NARG(...) \
PP_NARG_(__VA_ARGS__,PP_RSEQ_N())
#define PP_NARG_(...) \
PP_ARG_N(__VA_ARGS__)
#define PP_ARG_N( \
_1, _2, _3, _4, _5, _6, _7, _8, _9, N, ...) N
#define PP_RSEQ_N() 9,8,7,6,5,4,3,2,1,0
// support max 5 test func now
#define FN_NAME_SET_1(a) {#a}
#define FN_NAME_SET_2(a, b) {#a, #b}
#define FN_NAME_SET_3(a, b, c) {#a, #b, #c}
#define FN_NAME_SET_4(a, b, c, d) {#a, #b, #c, #d}
#define FN_NAME_SET_5(a, b, c, d, e) {#a, #b, #c, #d, #e}
#define FN_NAME_SET2(n) FN_NAME_SET_##n
#define FN_NAME_SET(n, ...) FN_NAME_SET2(n)(__VA_ARGS__)
#define UNITY_TEST_FN_SET(...) \
static test_func UNITY_TEST_UID(test_functions)[] = {__VA_ARGS__}; \
static char* UNITY_TEST_UID(test_fn_name)[] = FN_NAME_SET(PP_NARG(__VA_ARGS__), __VA_ARGS__)
typedef void (* test_func)(void);
struct test_desc_t
{
const char* name;
const char* desc;
void (*fn)(void);
const char* file;
int line;
struct test_desc_t* next;
const char* name;
const char* desc;
test_func* fn;
const char* file;
int line;
uint8_t test_fn_count;
char ** test_fn_name;
struct test_desc_t* next;
};
void unity_testcase_register(struct test_desc_t* desc);
@ -46,7 +75,7 @@ void unity_run_tests_with_filter(const char* filter);
void unity_run_all_tests();
/* Test case macro, a-la CATCH framework.
First argument is a free-form description,
First argument is a free-form description,
second argument is (by convention) a list of identifiers, each one in square brackets.
Identifiers are used to group related tests, or tests with specific properties.
Use like:
@ -56,21 +85,51 @@ void unity_run_all_tests();
// test goes here
}
*/
#define TEST_CASE(name_, desc_) \
static void UNITY_TEST_UID(test_func_) (void); \
static void __attribute__((constructor)) UNITY_TEST_UID(test_reg_helper_) () \
{ \
static struct test_desc_t UNITY_TEST_UID(test_desc_) = { \
.name = name_, \
.desc = desc_, \
.fn = &UNITY_TEST_UID(test_func_), \
.file = __FILE__, \
.line = __LINE__, \
.next = NULL \
}; \
unity_testcase_register( & UNITY_TEST_UID(test_desc_) ); \
}\
static void UNITY_TEST_UID(test_func_) (void)
static void UNITY_TEST_UID(test_func_) (void); \
static void __attribute__((constructor)) UNITY_TEST_UID(test_reg_helper_) () \
{ \
static test_func test_fn_[] = {&UNITY_TEST_UID(test_func_)}; \
static struct test_desc_t UNITY_TEST_UID(test_desc_) = { \
.name = name_, \
.desc = desc_, \
.fn = test_fn_, \
.file = __FILE__, \
.line = __LINE__, \
.test_fn_count = 1, \
.test_fn_name = NULL, \
.next = NULL \
}; \
unity_testcase_register( & UNITY_TEST_UID(test_desc_) ); \
}\
static void UNITY_TEST_UID(test_func_) (void)
/*
* First argument is a free-form description,
* second argument is (by convention) a list of identifiers, each one in square brackets.
* subsequent arguments are names of test functions for different DUTs
* e.g:
* TEST_CASE_MULTIPLE_DEVICES("master and slave spi","[spi][test_env=UT_T2_1]", master_test, slave_test);
* */
#define TEST_CASE_MULTIPLE_DEVICES(name_, desc_, ...) \
UNITY_TEST_FN_SET(__VA_ARGS__); \
static void __attribute__((constructor)) UNITY_TEST_UID(test_reg_helper_) () \
{ \
static struct test_desc_t UNITY_TEST_UID(test_desc_) = { \
.name = name_, \
.desc = desc_, \
.fn = UNITY_TEST_UID(test_functions), \
.file = __FILE__, \
.line = __LINE__, \
.test_fn_count = PP_NARG(__VA_ARGS__), \
.test_fn_name = UNITY_TEST_UID(test_fn_name), \
.next = NULL \
}; \
unity_testcase_register( & UNITY_TEST_UID(test_desc_) ); \
}
/**
* Note: initialization of test_desc_t fields above has to be done exactly
* in the same order as the fields are declared in the structure.

31
tools/unit-test-app/components/unity/test_utils.c
View file

@ -12,8 +12,11 @@
// See the License for the specific language governing permissions and
// limitations under the License.
#include <string.h>
#include "unity.h"
#include "test_utils.h"
#include "rom/ets_sys.h"
#include "rom/uart.h"
const esp_partition_t *get_test_data_partition()
{
@ -23,3 +26,31 @@ const esp_partition_t *get_test_data_partition()
TEST_ASSERT_NOT_NULL(result); /* means partition table set wrong */
return result;
}
// wait user to send "Enter" key
static void wait_user_control()
{
char sign[5] = {0};
while(strlen(sign) == 0)
{
/* Flush anything already in the RX buffer */
while(uart_rx_one_char((uint8_t *) sign) == OK) {
}
/* Read line */
UartRxString((uint8_t*) sign, sizeof(sign) - 1);
}
}
// signal functions, used for sync between unity DUTs for multiple devices cases
void unity_wait_for_signal(const char* signal_name)
{
printf("Waiting for signal: [%s]!\n"
"Please press \"Enter\" key to once any board send this signal.\n", signal_name);
wait_user_control();
}
void unity_send_signal(const char* signal_name)
{
printf("Send signal: [%s]!\n", signal_name);
}

61
tools/unit-test-app/components/unity/unity_platform.c
View file

@ -145,12 +145,57 @@ void unity_testcase_register(struct test_desc_t* desc)
}
}
/* print the multiple devices case name and its sub-menu
* e.g:
* (1) spi master/slave case
* (1)master case
* (2)slave case
* */
static void print_multiple_devices_test_menu(const struct test_desc_t* test_ms)
{
unity_printf("%s\n", test_ms->name);
for (int i = 0; i < test_ms->test_fn_count; i++)
{
unity_printf("\t(%d)\t\"%s\"\n", i+1, test_ms->test_fn_name[i]);
}
}
void multiple_devices_option(const struct test_desc_t* test_ms)
{
int selection;
char cmdline[256] = {0};
print_multiple_devices_test_menu(test_ms);
while(strlen(cmdline) == 0)
{
/* Flush anything already in the RX buffer */
while(uart_rx_one_char((uint8_t *) cmdline) == OK) {
}
UartRxString((uint8_t*) cmdline, sizeof(cmdline) - 1);
if(strlen(cmdline) == 0) {
/* if input was newline, print a new menu */
print_multiple_devices_test_menu(test_ms);
}
}
selection = atoi((const char *) cmdline) - 1;
if(selection >= 0 && selection < test_ms->test_fn_count) {
UnityDefaultTestRun(test_ms->fn[selection], test_ms->name, test_ms->line);
} else {
printf("Invalid selection, your should input number 1-%d!", test_ms->test_fn_count);
}
}
static void unity_run_single_test(const struct test_desc_t* test)
{
printf("Running %s...\n", test->name);
Unity.TestFile = test->file;
Unity.CurrentDetail1 = test->desc;
UnityDefaultTestRun(test->fn, test->name, test->line);
if(test->test_fn_count == 1) {
UnityDefaultTestRun(test->fn[0], test->name, test->line);
} else {
multiple_devices_option(test);
}
}
static void unity_run_single_test_by_index(int index)
@ -158,6 +203,7 @@ static void unity_run_single_test_by_index(int index)
const struct test_desc_t* test;
for (test = s_unity_tests_first; test != NULL && index != 0; test = test->next, --index)
{
}
if (test != NULL)
{
@ -201,7 +247,7 @@ static void unity_run_single_test_by_name(const char* filter)
{
unity_run_single_test(test);
}
}
}
}
void unity_run_all_tests()
@ -253,8 +299,15 @@ static int print_test_menu(void)
test = test->next, ++test_counter)
{
unity_printf("(%d)\t\"%s\" %s\n", test_counter + 1, test->name, test->desc);
}
return test_counter;
if(test->test_fn_count > 1)
{
for (int i = 0; i < test->test_fn_count; i++)
{
unity_printf("\t(%d)\t\"%s\"\n", i+1, test->test_fn_name[i]);
}
}
}
return test_counter;
}
static int get_test_count(void)