ehci: add ChipIdea isochronous transfers Opt in with CFG_TUH_CHIPIDEA_ISO_ENABLE (about 2.2 KiB without hub support); the audio_host example enables it and now builds for LPC43 and i.MX RT. Supported: HS endpoints with up to three transactions per microframe, FS endpoints on a native FS port, and FS endpoints behind a HS hub through split transactions. - One outstanding transfer and one iTD/siTD per endpoint, sharing the QH and qTD pools. Completion unlinks the TD before USBH gets the buffer back, so the class callback can submit the next interval. - No SOF interrupt (ChipIdea raises one every 125 us, even on a FS port): completions drive the scheduler; frame-list rollover arms TDs beyond the 8-frame ring, retires slots that passed unvisited and keeps the extended microframe counter current. FRINDEX is rechecked just before linking, so a delayed submission moves to a later slot of its phase. A TD is published for the next microframe (ChipIdea caches no descriptors). - Missed slots, cancellation and close retire a TD only when the controller cannot be on it (iso_td_idle), without stopping the periodic schedule. Schedule waits are bounded; bulk/interrupt abort keeps the transfer when the schedule does not stop. - FS behind a HS hub reserves best-effort MTT windows per hub port; split code is compiled out without CFG_TUH_HUB. - EHCI descriptors and the frame list move to CFG_TUH_UNCACHED_MEM_SECTION with barriers instead of descriptor cache maintenance. - Endpoint lookup by address, idempotent ISO reopen, named iTD/siTD masks. Host-driver fixtures run under CTest and pre-commit: generic EHCI, ISO disabled/enabled, cached payloads and no hub. The deferred USBH abort and endpoint-lifetime work is recorded in docs/superpowers/followup.
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index fdff4f8..60ede14 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml
@@ -55,6 +55,13 @@ types_or: [c, header] language: system + - id: host-test + name: host driver tests (CTest) + files: ^(src/|test/unit-test/|\.pre-commit-config\.yaml$) + entry: sh -c "cmake -S test/unit-test/host -B build/host-test -DCMAKE_BUILD_TYPE=Debug && cmake --build build/host-test --parallel 2 && ctest --test-dir build/host-test --output-on-failure" + pass_filenames: false + language: system + # Two hooks, split by what each suite RUNS, not by what it reads: discovery is # disjoint (test_hil*.py vs the two named suites) so nothing runs twice, but the # file patterns overlap where both suites care. hil-test runs test_hil*.py only
diff --git a/docs/superpowers/followup/pr3991-usbh-abort-lifetime.md b/docs/superpowers/followup/pr3991-usbh-abort-lifetime.md new file mode 100644 index 0000000..66c523c --- /dev/null +++ b/docs/superpowers/followup/pr3991-usbh-abort-lifetime.md
@@ -0,0 +1,96 @@ +# USBH abort and endpoint lifetime follow-up plan + +**Origin:** [PR #3991](https://github.com/hathach/tinyusb/pull/3991). + +**Goal:** Define and enforce endpoint ownership when cancellation fails or a +completion is pending during endpoint close and reuse. + +**Architecture:** Address this in common USBH and its HCD contract, with native +fake-HCD regression tests. Keep the ChipIdea ISO scheduler independent of this +work. Choose the reuse policy before implementing cross-OSAL synchronization. + +**Tech stack:** C99, TinyUSB USBH/HCD, OSAL queues, CMake/CTest. + +## Why deferred + +The maintainer requested that failed-abort handling also be deferred. PR #3991 +therefore retains the previous USBH abort/close behavior. The attempted ownership +fix and its fixture are removed; the macOS EHCI fixture skip remains. + +These are existing common-stack limitations. Changing ownership after abort +affects HCDs with unimplemented cancellation and HCDs with asynchronous retirement, +and requires a separate contract review and validation campaign. + +## Evidence + +- `src/host/usbh.c`: non-control `tuh_edpt_abort_xfer()` ignores the HCD return, + clears BUSY/CLAIMED, and returns true. EP0 also ignores the HCD return. +- OHCI, RP2040, MUSB, and RUSB2 return false from their unimplemented abort paths. + EHCI bulk/interrupt returns false if the descriptor has already completed. +- DWC2 periodic DMA accepts cancellation but rejects replacement submissions + until the halt ISR retires the channel and suppresses its completion. +- `tuh_task_ext()` dispatches a completion through the endpoint's current + callback after clearing its current ownership. An event already queued in + USBH is outside the HCD's cancellation mechanism. +- A native reproduction at `6c3bf1259` queued an old completion of 16 bytes, + closed/reopened the endpoint, submitted a new transfer with different callback + data, then ran the USBH task. The new callback received the old 16-byte result, + and the new transfer's BUSY flag was cleared. Source inspection shows that the + previous abort/close path also allowed this reuse; the lifetime problem was + not introduced by the ownership fix. +- Reviews: [failed abort](https://github.com/hathach/tinyusb/pull/3991#discussion_r4112166280), + [queued completion](https://github.com/hathach/tinyusb/pull/3991#discussion_r4112231165), + [close/reopen regression](https://github.com/hathach/tinyusb/pull/3991#discussion_r4112236957). + +## Remaining work + +### 1. Recover and extend the regression fixture in a separate branch + +Files: `test/unit-test/host/usbh/test_abort.c`, its `tusb_config.h`, and +`test/unit-test/host/CMakeLists.txt`. + +- [ ] Recover the fake-HCD fixture and target registration from commit + `6c3bf1259`; the attempted production fix is in `9d77cb78a`. +- [ ] Confirm its failed-abort assertion fails against the restored USBH code. +- [ ] Add IN and OUT cases for this sequence: submit A, queue A's completion, + close, reopen, attempt B, process the old event. Assert that A never invokes + B's callback or clears ownership of an accepted B. If reuse is deferred, + assert that B can be submitted after old-event retirement. +- [ ] Cover failed close, successful cancellation, delayed cancellation, + unrelated queued events, and class callbacks as well as application callbacks. + +Run the native fixtures with: + +```sh +cmake -S test/unit-test/host -B build/host-test -DCMAKE_BUILD_TYPE=Debug +cmake --build build/host-test +ctest --test-dir build/host-test --output-on-failure +``` + +### 2. Settle the common contract before changing implementation + +Files: `src/host/hcd.h`, `src/host/usbh.h`, `src/host/usbh.c`, and affected HCDs. + +- [ ] Distinguish unsuccessful cancellation, accepted deferred cancellation, + completed-but-undispatched transfers, and successful endpoint closure. +- [ ] Specify when buffers and endpoint storage can be reused, what callbacks + can still occur, and whether immediate close/reopen submission is supported. +- [ ] Review the unchanged EP0 path separately rather than assuming the + non-control ownership rule applies to control transfers. +- [ ] Evaluate retirement using existing endpoint-state bits before adding + memory or public APIs. The maintainer previously rejected an ISO-specific + generation counter; no replacement design is approved by this handoff. + +### 3. Implement and validate the agreed policy + +- [ ] Make the native regression cases pass without dropping unrelated events. +- [ ] If queue filtering is chosen, implement synchronization for supported + OSAL backends: there is currently no selective-removal API, and OSAL-none + queue receive/send can re-enable interrupts. Do not call `tuh_task()` + recursively from endpoint close. +- [ ] Check memory and code-size impact, run pre-commit, and build the complete + example sets for representative EHCI and other affected HCD boards. +- [ ] Exercise abort/close/reopen while transfers are active on hardware, + including an HCD that cannot cancel and one with deferred retirement. +- [ ] Open a separate USBH PR referencing #3991 and the reproduced sequence. +- [ ] Delete this handoff when that PR lands.
diff --git a/examples/host/audio_host/only.txt b/examples/host/audio_host/only.txt index becebd3..8105108 100644 --- a/examples/host/audio_host/only.txt +++ b/examples/host/audio_host/only.txt
@@ -28,3 +28,5 @@ mcu:STM32N6 mcu:STM32U5 mcu:STM32WBA +mcu:LPC43XX +mcu:MIMXRT1XXX
diff --git a/examples/host/audio_host/src/tusb_config.h b/examples/host/audio_host/src/tusb_config.h index ca3ec0e..595f8eb 100644 --- a/examples/host/audio_host/src/tusb_config.h +++ b/examples/host/audio_host/src/tusb_config.h
@@ -95,6 +95,9 @@ #define CFG_TUH_DEVICE_MAX (3 * CFG_TUH_HUB + 1) //------------- Audio Host Config -------------// +#ifndef CFG_TUH_CHIPIDEA_ISO_ENABLE + #define CFG_TUH_CHIPIDEA_ISO_ENABLE 1 +#endif #define CFG_TUH_AUDIO_EPIN_BUFSIZE 256 // max capture transfer the application submits #define CFG_TUH_AUDIO_EPOUT_BUFSIZE 256 // max playback transfer the application submits
diff --git a/src/portable/ehci/ehci.c b/src/portable/ehci/ehci.c index 0d38c9d..ba54599 100644 --- a/src/portable/ehci/ehci.c +++ b/src/portable/ehci/ehci.c
@@ -17,9 +17,22 @@ #include "host/hcd.h" #include "host/usbh.h" #include "host/usbh_pvt.h" +#if CFG_TUH_HUB +#include "host/hub.h" +#endif #include "ehci_api.h" #include "ehci.h" +#ifndef TU_MEMORY_BARRIER + #error "TU_MEMORY_BARRIER() is not defined for this compiler (see tusb_compiler.h)" +#endif + +// Descriptors and the frame list get no cache maintenance. With D-cache in use, +// their section must be confirmed non-cacheable (or coherent) explicitly. +#if CFG_TUH_MEM_DCACHE_ENABLE && defined(TUH_UNCACHED_MEM_SECTION_IMPLICIT) + #warning "EHCI descriptors use CFG_TUH_MEM_SECTION, which must be non-cacheable; define CFG_TUH_UNCACHED_MEM_SECTION to confirm" +#endif + // NXP specific fixes #if TU_CHECK_MCU(OPT_MCU_MIMXRT1XXX, OPT_MCU_LPC55, OPT_MCU_MCXN9, OPT_MCU_RW61X) #include "fsl_device_registers.h" @@ -45,7 +58,7 @@ #else // STD EHCI: 256 elements #define FRAMELIST_SIZE_BIT_VALUE 2u - #define FRAMELIST_SIZE_USBCMD_VALUE ((FRAMELIST_SIZE_BIT_VALUE & 3) << EHCI_USBCMD_POS_FRAMELIST_SIZE) + #define FRAMELIST_SIZE_USBCMD_VALUE ((FRAMELIST_SIZE_BIT_VALUE & 3) << EHCI_USBCMD_FRAMELIST_SIZE_SHIFT) #endif #define FRAMELIST_SIZE (1024 >> FRAMELIST_SIZE_BIT_VALUE) @@ -54,6 +67,107 @@ #define QHD_MAX (CFG_TUH_DEVICE_MAX*CFG_TUH_ENDPOINT_MAX + CFG_TUH_HUB) #define QTD_MAX QHD_MAX +/* Isochronous transfers (ChipIdea HS, CFG_TUH_CHIPIDEA_ISO_ENABLE) + * + * Each endpoint has one outstanding transfer covering one service interval + * and owns one TD: an iTD at HS (up to 3 transactions) or an siTD at FS. + * Larger requests are rejected. The TD occupies one QH pool slot and the + * endpoint state one qTD pool slot until close; bulk and interrupt transfers + * use the rest. Completion unlinks the TD before USBH gets the buffer back, + * so its callback can submit the next interval. + * + * Scheduling uses no SOF interrupt (ChipIdea raises one every 125 us, even on + * a FS port). Completions (IOC) drive it; frame-list rollover, once per + * 8-frame ring, arms TDs whose frame lies beyond the current ring, retires + * slots that passed unvisited and keeps the extended microframe counter + * current. + * - Endpoint phase is preserved and late slots are skipped. FRINDEX is + * rechecked just before linking, so a delayed submission moves to a later + * slot of its phase instead of publishing one the controller may have passed. + * - ChipIdea caches no descriptors (its isochronous scheduling threshold is + * always 0), so a TD is published for the next microframe, one less than + * the lead EHCI 4.7.2.1 asks for; this keeps 125 us HS intervals back to + * back and is verified on RT1064, not specified. + * - On a native FS port the counter lags FRINDEX by one frame + * (iso_frame_offset). The ChipIdea manual documents only a one-microframe + * B/H-frame lag; the frame offset is empirical. + * - A TD that is still not visited may be retired by rollover only after the + * ring repeats; it is then reported failed while its buffer stays owned. + * Missed slots, cancellation and close retire a TD only once the controller + * cannot be on it (iso_td_idle), without stopping the periodic schedule; + * close falls back to a bounded schedule stop. Missed packets are not retried. + * + * FS behind a HS hub uses best-effort MTT windows per (hub, port), reserved by + * maximum packet size: OUT from H0 upwards, IN from the end with two + * complete-split retries before H7 ends. No bus-time admission, interrupt + * reservations, single-TT accounting or frame-spanning splits; IN is limited + * to 564 bytes and OUT to 1023. Direct FS uses the embedded TT, which ignores + * the S-mask/C-mask (left zero) and allows 1023 bytes in either direction. + * + * CFG_TUH_UNCACHED_MEM_SECTION must be non-cacheable or coherent because TDs + * are unlinked while the schedule runs; payload buffers keep cache maintenance. + * + * References: EHCI 1.0 ch. 3/4; USB 2.0 ch. 11; RT1064 RM Rev. 2 ch. 42; + * ChipIdea (TDI) HS USB controller core manual. + */ +#if defined(TUP_USBIP_CHIPIDEA_HS) && CFG_TUH_CHIPIDEA_ISO_ENABLE + #define EHCI_ISO_ENABLED 1 +#else + #define EHCI_ISO_ENABLED 0 +#endif + +#if EHCI_ISO_ENABLED +// An iTD must not cross a 4 KiB boundary (EHCI chapter 3). +typedef union TU_ATTR_ALIGNED(64) { + ehci_itd_t itd; + ehci_sitd_t sitd; + volatile uint32_t words[16]; // snapshot a hardware status word without repeated bitfield reads +} iso_td_t; + +#define ITD_XACT_WORD(slot) (1u + (slot)) // iTD transaction status/control +#define SITD_EP_WORD 1u // siTD endpoint characteristics +#define SITD_STATE_WORD 3u // siTD transfer status/control + +typedef struct { + iso_td_t* td; + uint8_t* buffer; + uint32_t interval; + uint32_t uframe; // Current service slot while busy, next slot while idle. + uint16_t max_xfer_bytes; + uint16_t buflen; + uint8_t daddr; + uint8_t ep_addr; + uint8_t speed; + uint8_t split_slot; + bool busy; + bool armed; +} iso_ep_t; + +TU_VERIFY_STATIC(sizeof(iso_ep_t) <= sizeof(ehci_qtd_t), "ISO state must fit in one qTD slot"); +#endif + +typedef union { + ehci_qhd_t qhd; +#if EHCI_ISO_ENABLED + iso_td_t iso; +#endif +} ehci_ep_t; + +typedef union { + ehci_qtd_t qtd; +#if EHCI_ISO_ENABLED + iso_ep_t iso; +#endif +} ehci_td_t; + +// Exactly one member is non-NULL for an open endpoint. +typedef struct { + ehci_qhd_t* qhd; +#if EHCI_ISO_ENABLED + iso_ep_t* iso; +#endif +} ehci_edpt_t; + typedef struct { ehci_link_t period_framelist[FRAMELIST_SIZE]; @@ -68,8 +182,19 @@ ehci_qtd_t qtd; }control[CFG_TUH_DEVICE_MAX+CFG_TUH_HUB+1]; - ehci_qhd_t qhd_pool[QHD_MAX]; - ehci_qtd_t qtd_pool[QTD_MAX] TU_ATTR_ALIGNED(32); + ehci_ep_t qhd_pool[QHD_MAX]; + ehci_td_t qtd_pool[QTD_MAX] TU_ATTR_ALIGNED(32); +#if EHCI_ISO_ENABLED + uint32_t qhd_is_iso[(QHD_MAX + 31) / 32]; + uint32_t qtd_is_iso[(QTD_MAX + 31) / 32]; +#endif + +#if EHCI_ISO_ENABLED + uint32_t iso_uframe; + uint16_t iso_last_frindex; + uint8_t iso_saved_itc; + uint8_t iso_frame_offset; +#endif ehci_registers_t* regs; // operational register ehci_cap_registers_t* cap_regs; // capability register @@ -77,8 +202,30 @@ volatile uint32_t uframe_number; }ehci_data_t; -// Periodic frame list must be 4K alignment -CFG_TUH_MEM_SECTION TU_ATTR_ALIGNED(4096) static ehci_data_t ehci_data; +// Live descriptors/links must be uncached or DMA-coherent. The frame list must be 4K aligned. +CFG_TUH_UNCACHED_MEM_SECTION TU_ATTR_ALIGNED(4096) static ehci_data_t ehci_data; + +#if EHCI_ISO_ENABLED +TU_ATTR_ALWAYS_INLINE static inline bool qhd_is_iso(size_t index) { + return (ehci_data.qhd_is_iso[index / 32] & TU_BIT(index % 32)) != 0; +} + +TU_ATTR_ALWAYS_INLINE static inline bool qtd_is_iso(size_t index) { + return (ehci_data.qtd_is_iso[index / 32] & TU_BIT(index % 32)) != 0; +} + +// Skip unused pool slots without imposing a separate ISO endpoint limit. +static size_t iso_ep_next(size_t index) { + while (index < QTD_MAX) { + uint32_t const mask = ehci_data.qtd_is_iso[index / 32] >> (index % 32); + if (mask != 0) { + return index + tu_log2(mask & (0u - mask)); + } + index = (index | 31u) + 1; + } + return QTD_MAX; +} +#endif //--------------------------------------------------------------------+ // Debug @@ -125,8 +272,8 @@ TU_ATTR_ALWAYS_INLINE static inline ehci_qhd_t* qhd_control(uint8_t dev_addr); TU_ATTR_ALWAYS_INLINE static inline ehci_qhd_t* qhd_next (ehci_qhd_t const * p_qhd); -TU_ATTR_ALWAYS_INLINE static inline ehci_qhd_t* qhd_find_free (void); -static ehci_qhd_t* qhd_get_from_addr (uint8_t dev_addr, uint8_t ep_addr); +static ehci_qhd_t* qhd_find_free (void); +static ehci_edpt_t edpt_find(uint8_t dev_addr, uint8_t ep_addr); static void qhd_init(ehci_qhd_t *p_qhd, uint8_t dev_addr, tusb_desc_endpoint_t const * ep_desc); static void qhd_attach_qtd(ehci_qhd_t *qhd, ehci_qtd_t *qtd); static void qhd_remove_qtd(ehci_qhd_t *qhd); @@ -138,7 +285,7 @@ } TU_ATTR_ALWAYS_INLINE static inline ehci_qtd_t* qtd_control(uint8_t dev_addr); -TU_ATTR_ALWAYS_INLINE static inline ehci_qtd_t* qtd_find_free (void); +static ehci_qtd_t* qtd_find_free (void); static void qtd_init (ehci_qtd_t* qtd, void const* buffer, uint16_t total_bytes); TU_ATTR_ALWAYS_INLINE static inline ehci_link_t* list_get_period_head(uint8_t rhport, uint32_t interval_ms); @@ -148,25 +295,57 @@ TU_ATTR_ALWAYS_INLINE static inline void list_remove(ehci_link_t* head, ehci_link_t* prev, ehci_qhd_t* qhd); static void list_remove_qhd_by_addr(ehci_link_t *list_head, uint8_t dev_addr, uint8_t ep_addr); -static void ehci_disable_schedule(ehci_registers_t* regs, bool is_period) { - // maybe have a timeout for status - if (is_period) { - regs->command_bm.periodic_enable = 0; - while(regs->status_bm.periodic_status) {} - } else { - regs->command_bm.async_enable = 0; - while(regs->status_bm.async_status) {} // should have a timeout +#if EHCI_ISO_ENABLED +static bool iso_ep_open(uint8_t rhport, uint8_t daddr, tusb_desc_endpoint_t const* desc); +static bool iso_stop(uint8_t rhport, iso_ep_t* ep, bool close); +static bool iso_xfer(uint8_t rhport, iso_ep_t* ep, uint8_t* buffer, uint16_t buflen); +static void iso_process(bool in_isr); +#endif + +// Wait until a schedule's status bit equals on. The controller samples the +// enable bit only at (micro)frame boundaries (EHCI 4.6, 4.8), so the wait is +// bounded by FRINDEX (2 frames), by a halted controller, and by an iteration +// guard in case FRINDEX is not counting. +static bool ehci_schedule_wait(ehci_registers_t* regs, bool is_period, bool on) { + uint32_t const start = regs->frame_index; + for (uint32_t guard = 0; guard < 0x100000u; guard++) { + bool const status = is_period ? regs->status_bm.periodic_status : regs->status_bm.async_status; + if (status == on) { + return true; + } + if (regs->status_bm.hc_halted || ((regs->frame_index - start) & 0x3fffu) > 16u) { + return false; + } } + return false; } +// Stop a schedule and wait until the controller no longer runs it. The enable +// bit changes only once its status matches it (EHCI 4.6). Returns false if the +// schedule did not stop in time; a halted controller runs nothing. +static bool ehci_disable_schedule(ehci_registers_t* regs, bool is_period) { + if (regs->status_bm.hc_halted) { + return true; + } + bool const enabled = is_period ? regs->command_bm.periodic_enable : regs->command_bm.async_enable; + if (enabled && !ehci_schedule_wait(regs, is_period, true)) { + return false; + } + if (is_period) { + regs->command_bm.periodic_enable = 0; + } else { + regs->command_bm.async_enable = 0; + } + return ehci_schedule_wait(regs, is_period, false); +} + +// Restart a schedule without waiting for it to run; the next disable waits for +// the status to catch up before changing the enable bit again. static void ehci_enable_schedule(ehci_registers_t* regs, bool is_period) { - // maybe have a timeout for status if (is_period) { regs->command_bm.periodic_enable = 1; - while ( 0 == regs->status_bm.periodic_status ) {} } else { regs->command_bm.async_enable = 1; - while( 0 == regs->status_bm.async_status ) {} } } @@ -276,6 +455,14 @@ return; } +#if EHCI_ISO_ENABLED + for (size_t i = iso_ep_next(0); i < QTD_MAX; i = iso_ep_next(i + 1)) { + if (ehci_data.qtd_pool[i].iso.daddr == daddr) { + (void) iso_stop(rhport, &ehci_data.qtd_pool[i].iso, true); + } + } +#endif + // Remove from async list all endpoints of this device list_remove_qhd_by_addr((ehci_link_t *) list_get_async_head(rhport), daddr, TUSB_INDEX_INVALID_8); @@ -368,7 +555,7 @@ init_periodic_list(rhport); regs->periodic_list_base = (uint32_t) ehci_data.period_framelist; - hcd_dcache_clean(&ehci_data, sizeof(ehci_data_t)); + TU_MEMORY_BARRIER(); //------------- TT Control (NXP only) -------------// regs->nxp_tt_control = 0; @@ -414,15 +601,20 @@ //--------------------------------------------------------------------+ bool hcd_edpt_open(uint8_t rhport, uint8_t dev_addr, tusb_desc_endpoint_t const * ep_desc) { - // TODO not support ISO yet - TU_ASSERT (ep_desc->bmAttributes.xfer != TUSB_XFER_ISOCHRONOUS); + if (ep_desc->bmAttributes.xfer == TUSB_XFER_ISOCHRONOUS) { +#if EHCI_ISO_ENABLED + return iso_ep_open(rhport, dev_addr, ep_desc); +#else + return false; +#endif + } //------------- Prepare Queue Head -------------// ehci_qhd_t *p_qhd; if (ep_desc->bEndpointAddress == 0) { p_qhd = qhd_control(dev_addr); } else { - if (NULL != qhd_get_from_addr(dev_addr, ep_desc->bEndpointAddress)) { + if (NULL != edpt_find(dev_addr, ep_desc->bEndpointAddress).qhd) { return true; // already opened } p_qhd = qhd_find_free(); @@ -447,10 +639,6 @@ list_head = list_get_period_head(rhport, p_qhd->interval_ms); break; - case TUSB_XFER_ISOCHRONOUS: - // TODO iso is not supported - break; - default: break; } @@ -458,14 +646,18 @@ list_insert(list_head, (ehci_link_t*) p_qhd, EHCI_QTYPE_QHD); - hcd_dcache_clean(p_qhd, sizeof(ehci_qhd_t)); - hcd_dcache_clean(list_head, sizeof(ehci_qhd_t)); - return true; } bool hcd_edpt_close(uint8_t rhport, uint8_t daddr, uint8_t ep_addr) { - ehci_qhd_t* qhd = qhd_get_from_addr(daddr, ep_addr); + ehci_edpt_t const ep = edpt_find(daddr, ep_addr); +#if EHCI_ISO_ENABLED + iso_ep_t* iso = ep.iso; + if (iso != NULL) { + return iso_stop(rhport, iso, true); + } +#endif + ehci_qhd_t* qhd = ep.qhd; TU_VERIFY(qhd != NULL); ehci_link_t * list_head; @@ -505,10 +697,18 @@ bool hcd_edpt_xfer(uint8_t rhport, uint8_t dev_addr, uint8_t ep_addr, uint8_t * buffer, uint16_t buflen) { (void) rhport; + ehci_edpt_t const ep = edpt_find(dev_addr, ep_addr); +#if EHCI_ISO_ENABLED + iso_ep_t* iso = ep.iso; + if (iso != NULL) { + return iso_xfer(rhport, iso, buffer, buflen); + } +#endif + uint8_t const epnum = tu_edpt_number(ep_addr); uint8_t const dir = tu_edpt_dir(ep_addr); - ehci_qhd_t* qhd = qhd_get_from_addr(dev_addr, ep_addr); + ehci_qhd_t* qhd = ep.qhd; TU_VERIFY(qhd != NULL); ehci_qtd_t* qtd; @@ -551,30 +751,40 @@ bool hcd_edpt_abort_xfer(uint8_t rhport, uint8_t dev_addr, uint8_t ep_addr) { (void) rhport; - // TODO ISO not supported yet - ehci_qhd_t* qhd = qhd_get_from_addr(dev_addr, ep_addr); + ehci_edpt_t const ep = edpt_find(dev_addr, ep_addr); +#if EHCI_ISO_ENABLED + iso_ep_t* iso = ep.iso; + if (iso != NULL) { + return iso_stop(rhport, iso, false); + } +#endif + ehci_qhd_t* qhd = ep.qhd; + TU_VERIFY(qhd != NULL); ehci_qtd_t * volatile qtd = qhd->attached_qtd; TU_VERIFY(qtd != NULL); // no queued transfer - hcd_dcache_invalidate(qtd, sizeof(ehci_qtd_t)); TU_VERIFY(qtd->active); // transfer is already complete // HC is still processing, disable HC list schedule before making changes bool const is_period = (qhd->interval_ms > 0); - ehci_disable_schedule(ehci_data.regs, is_period); + if (!ehci_disable_schedule(ehci_data.regs, is_period)) { + // The controller may still own the TD: leave the transfer in place. + ehci_enable_schedule(ehci_data.regs, is_period); + return false; + } // check active bit again just in case HC has just processed the TD bool const still_active = qtd->active; if (still_active) { // remove TD from QH overlay qhd->qtd_overlay.next.terminate = 1; - hcd_dcache_clean(qhd, sizeof(ehci_qhd_t)); // remove TD from QH software list qhd_remove_qtd(qhd); } + TU_MEMORY_BARRIER(); ehci_enable_schedule(ehci_data.regs, is_period); return still_active; // true if removed an active transfer @@ -582,14 +792,446 @@ bool hcd_edpt_clear_stall(uint8_t rhport, uint8_t daddr, uint8_t ep_addr) { (void) rhport; - ehci_qhd_t *qhd = qhd_get_from_addr(daddr, ep_addr); + ehci_qhd_t *qhd = edpt_find(daddr, ep_addr).qhd; + TU_VERIFY(qhd != NULL); qhd->qtd_overlay.halted = 0; qhd->qtd_overlay.data_toggle = 0; - hcd_dcache_clean_invalidate(qhd, sizeof(ehci_qhd_t)); + TU_MEMORY_BARRIER(); return true; } +#if EHCI_ISO_ENABLED +//--------------------------------------------------------------------+ +// Isochronous transfers: one service interval per HCD submission +//--------------------------------------------------------------------+ + +// FRINDEX is a 14-bit microframe counter, independent of the frame-list size +// (RT1064 RM 42.7.21). Sample at each SOF, including while an endpoint is idle. +static uint32_t iso_now(void) { + uint16_t const index = (uint16_t) (ehci_data.regs->frame_index & 0x3fff); + ehci_data.iso_uframe += (index - ehci_data.iso_last_frindex) & 0x3fff; + ehci_data.iso_last_frindex = index; + // Native FS runs the embedded translator on frame boundaries. FRINDEX has + // already advanced to the next frame while the current bus frame executes. + // On a HS root link the normal microframe scheduling clock applies. + return ehci_data.iso_uframe - ehci_data.iso_frame_offset; +} + +// First slot that can still be published: the next microframe (see above). +static uint32_t iso_earliest(uint32_t now) { + return now + 1; +} + +static void iso_ep_free(iso_ep_t* ep) { + size_t const qhd_index = (ehci_ep_t*) ep->td - ehci_data.qhd_pool; + size_t const qtd_index = (ehci_td_t*) ep - ehci_data.qtd_pool; + tu_memclr(ep->td, sizeof(iso_td_t)); + tu_memclr(ep, sizeof(*ep)); + ehci_data.qhd_is_iso[qhd_index / 32] &= ~TU_BIT(qhd_index % 32); + ehci_data.qtd_is_iso[qtd_index / 32] &= ~TU_BIT(qtd_index % 32); +} + +// Reserve by resolved upstream HS hub/port, assuming a multi-TT hub. +#if CFG_TUH_HUB +static bool iso_split_slot(uint8_t hub_addr, uint8_t hub_port, uint16_t packet_size, uint8_t dir, uint8_t* slot) { + uint8_t used = 0; + for (size_t i = iso_ep_next(0); i < QTD_MAX; i = iso_ep_next(i + 1)) { + iso_ep_t const* ep = &ehci_data.qtd_pool[i].iso; + ehci_sitd_t const* td = &ep->td->sitd; + if (ep->speed == TUSB_SPEED_FULL && td->hub_addr == hub_addr && td->port_number == hub_port) { + uint8_t const count = (uint8_t) ((ep->max_xfer_bytes + 187u) / 188u); + used |= ((1u << count) - 1u) << ep->split_slot; + } + } + return hub_iso_split_slot(used, packet_size, dir, slot); +} +#endif + +static bool iso_ep_open(uint8_t rhport, uint8_t daddr, tusb_desc_endpoint_t const* desc) { + TU_VERIFY(daddr != 0 && tu_edpt_number(desc->bEndpointAddress) != 0); + tuh_bus_info_t bus; + TU_VERIFY(tuh_bus_info_get(daddr, &bus)); + TU_VERIFY(bus.speed == TUSB_SPEED_FULL || bus.speed == TUSB_SPEED_HIGH); + TU_VERIFY(desc->bInterval >= 1 && desc->bInterval <= 16); + uint16_t const mps = tu_edpt_packet_size(desc); + uint8_t const mult = (uint8_t) (((tu_le16toh(desc->wMaxPacketSize) >> 11) & 3) + 1); + TU_VERIFY(mps != 0 && mult <= 3); + if (bus.speed == TUSB_SPEED_FULL) { + TU_VERIFY(mps <= 1023 && mult == 1); + } else { + TU_VERIFY(mps <= 1024); + } + uint32_t const interval = (1u << (desc->bInterval - 1)) * (bus.speed == TUSB_SPEED_FULL ? 8u : 1u); + iso_ep_t const* opened = edpt_find(daddr, desc->bEndpointAddress).iso; + if (opened != NULL) { + // Reopening with other parameters needs a close first, which releases the + // TD and any split window sized for the old maximum packet. + return opened->max_xfer_bytes == mps * mult && opened->interval == interval; + } + + uint8_t const dir = tu_edpt_dir(desc->bEndpointAddress); + uint8_t hub_addr = 0; + uint8_t hub_port = 0; + uint8_t split_slot = 0; +#if CFG_TUH_HUB + // Find the upstream HS transaction translator, including FS hubs between + // the endpoint and TT. Direct FS uses the embedded TT (hub address zero). + hub_addr = bus.hub_addr; + hub_port = bus.hub_port; + while (bus.speed != TUSB_SPEED_HIGH && hub_addr != 0) { + tuh_bus_info_t hub; + TU_VERIFY(tuh_bus_info_get(hub_addr, &hub)); + if (hub.speed == TUSB_SPEED_HIGH) { + break; + } + hub_addr = hub.hub_addr; + hub_port = hub.hub_port; + } + bool const split = bus.speed == TUSB_SPEED_FULL && + ehci_data.regs->portsc_bm.nxp_port_speed == TUSB_SPEED_HIGH; + if (split) { + // Best effort, single H-frame only. Large FS IN packets require a + // frame-spanning complete-split schedule (RT1064 RM 42.5.3.12.3.1). + TU_VERIFY(dir == TUSB_DIR_OUT || mps <= 564); + TU_VERIFY(iso_split_slot(hub_addr, hub_port, mps, dir, &split_slot)); + } +#else + // Without hubs a FS endpoint is on a native FS port: no split transactions. + bool const split = false; +#endif + + hcd_int_disable(rhport); + ehci_qhd_t* qhd = qhd_find_free(); + ehci_qtd_t* qtd = qtd_find_free(); + if (qhd == NULL || qtd == NULL) { + hcd_int_enable(rhport); + return false; + } + size_t const qhd_index = ((uintptr_t) qhd - (uintptr_t) ehci_data.qhd_pool) / sizeof(ehci_ep_t); + size_t const qtd_index = (ehci_td_t*) qtd - ehci_data.qtd_pool; + bool const first_iso = iso_ep_next(0) == QTD_MAX; + iso_ep_t* ep = &ehci_data.qtd_pool[qtd_index].iso; + tu_memclr(ep, sizeof(*ep)); + ep->td = &ehci_data.qhd_pool[qhd_index].iso; + tu_memclr(ep->td, sizeof(iso_td_t)); + ehci_data.qhd_is_iso[qhd_index / 32] |= TU_BIT(qhd_index % 32); + ehci_data.qtd_is_iso[qtd_index / 32] |= TU_BIT(qtd_index % 32); + ep->ep_addr = desc->bEndpointAddress; + ep->speed = bus.speed; + ep->split_slot = split_slot; + ep->max_xfer_bytes = (uint16_t) (mps * mult); + ep->interval = interval; + // Keep the extended counter congruent to FRINDEX even when ISO was disabled. + if (first_iso) { + ehci_data.iso_last_frindex = (uint16_t) (ehci_data.regs->frame_index & 0x3fff); + ehci_data.iso_uframe = ehci_data.iso_last_frindex; + ehci_data.iso_saved_itc = ehci_data.regs->command_bm.int_threshold; + // Fixed for this root connection; refreshed when ISO is reopened. + ehci_data.iso_frame_offset = ehci_data.regs->portsc_bm.nxp_port_speed == TUSB_SPEED_FULL ? 8 : 0; + } + ep->uframe = iso_now(); + ep->daddr = daddr; + // Retire each packet promptly so task context can submit the next interval. + ehci_data.regs->command_bm.int_threshold = 0; + + iso_td_t* td = ep->td; + td->itd.next.terminate = 1; + if (ep->speed == TUSB_SPEED_HIGH) { + // Endpoint fields share the low bits of the buffer page pointers. + td->itd.BufferPointer[0] = ep->daddr | (tu_edpt_number(ep->ep_addr) << 8); + td->itd.BufferPointer[1] = mps | (dir << 11); + td->itd.BufferPointer[2] = mult; + } else { + // siTD endpoint characteristics (EHCI 3.4.2); the TD is not linked yet. + td->words[SITD_EP_WORD] = (ep->daddr & 0x7fu) | + ((uint32_t) tu_edpt_number(ep->ep_addr) << EHCI_SITD_EP_ENDPT_SHIFT) | + ((uint32_t) (hub_addr & 0x7fu) << EHCI_SITD_EP_HUB_SHIFT) | + ((uint32_t) (hub_port & 0x7fu) << EHCI_SITD_EP_PORT_SHIFT) | + ((uint32_t) dir << EHCI_SITD_EP_DIR_SHIFT); + td->sitd.back.terminate = 1; + if (dir == TUSB_DIR_IN && split) { + uint8_t const count = (uint8_t) ((mps + 187u) / 188u); + td->sitd.int_smask = 1u << split_slot; + td->sitd.fl_int_cmask = ((1u << (count + 2u)) - 1u) << (split_slot + 2u); + } + } + hcd_int_enable(rhport); + return true; +} + +// The caller has observed completion or stopped periodic DMA. A predecessor +// may still be active, so its link must be in coherent/non-cacheable memory. +static void iso_td_unlink(iso_ep_t const* ep) { + iso_td_t* td = ep->td; + ehci_link_t* link = &ehci_data.period_framelist[(ep->uframe >> 3) % FRAMELIST_SIZE]; + while (!link->terminate && tu_align32(link->address) != (uint32_t) td) { + link = list_next(link); + } + if (!link->terminate) { + *link = td->itd.next; + } + // Retire the single transaction, including cancelled or timed-out work. + TU_MEMORY_BARRIER(); + td->words[ep->speed == TUSB_SPEED_HIGH ? ITD_XACT_WORD(ep->uframe & 7) : SITD_STATE_WORD] = 0; + TU_MEMORY_BARRIER(); +} + +// True when the controller can neither be accessing ep's TD nor start before a +// single store retires it. The TD sits in frame-list entry (uframe >> 3) and is +// visited when FRINDEX reaches that frame. An iTD transaction runs only in its +// own microframe. An siTD runs during its H-frame, and on a native FS port the +// embedded TT works on it while FRINDEX already shows the next frame (see +// iso_frame_offset). One microframe or frame on each side is kept as margin. +static bool iso_td_idle(iso_ep_t const* ep) { + ehci_registers_t* regs = ehci_data.regs; + if (regs->status_bm.hc_halted) { + return true; + } + uint32_t const raw = regs->frame_index; + if (ep->speed == TUSB_SPEED_HIGH) { + uint32_t const d = (raw - ep->uframe) % (FRAMELIST_SIZE * 8u); + return d != 0 && d != FRAMELIST_SIZE * 8u - 1; + } + uint32_t const df = ((raw >> 3) - (ep->uframe >> 3)) % FRAMELIST_SIZE; + return df >= 3 && df <= FRAMELIST_SIZE - 2; +} + +// Advance to the endpoint's first slot that can still be published, keeping +// its phase: ISO intervals are powers of two. +static void iso_schedule(iso_ep_t* ep, uint32_t now) { + uint32_t earliest = iso_earliest(now); + if (ep->speed == TUSB_SPEED_FULL) { + earliest = (earliest + 7) & ~7u; + } + if ((int32_t) (ep->uframe - earliest) < 0) { + uint32_t const behind = earliest - ep->uframe; + ep->uframe += (behind + ep->interval - 1) & ~(ep->interval - 1); + } + if (ep->speed == TUSB_SPEED_FULL) { + ep->uframe = (ep->uframe + 7) & ~7u; + } +} + +// Called with the controller interrupt excluded. A frame-list entry may hold a +// new TD once its previous visit has ended, so the TD's frame must lie within +// the ring starting at the current frame; frame-list rollover arms the rest. +static void iso_arm(iso_ep_t* ep) { + iso_td_t* td = ep->td; + uint32_t const buffer = (uint32_t) ep->buffer; + uint32_t const page = buffer & ~0xfffu; + uint8_t const dir = tu_edpt_dir(ep->ep_addr); + for (;;) { + uint32_t const now = iso_now(); + iso_schedule(ep, now); + if ((ep->uframe & ~7u) - (now & ~7u) >= FRAMELIST_SIZE * 8u) { + return; + } + if (ep->speed == TUSB_SPEED_HIGH) { + ehci_itd_t* itd = &td->itd; + // One service interval is at most 3072 bytes, spanning at most two pages. + // BufferPointer[2] keeps only the endpoint's Mult field set at open. + itd->BufferPointer[0] = page | (itd->BufferPointer[0] & 0xfff); + itd->BufferPointer[1] = (page + 4096) | (itd->BufferPointer[1] & 0xfff); + uint8_t const slot = ep->uframe & 7; + td->words[ITD_XACT_WORD(slot)] = (buffer & 0xfff) | EHCI_ITD_XACT_IOC | + ((uint32_t) ep->buflen << EHCI_ITD_XACT_LENGTH_SHIFT); + } else { + ehci_sitd_t* sitd = &td->sitd; + sitd->buffer[0] = buffer; + sitd->buffer[1] = page + 4096; +#if CFG_TUH_HUB + if (dir == TUSB_DIR_OUT && !ehci_data.iso_frame_offset) { + uint8_t const count = (uint8_t) tu_max32(1, (ep->buflen + 187u) / 188u); + sitd->int_smask = (uint8_t) (((1u << count) - 1u) << ep->split_slot); + sitd->buffer[1] |= count | (count > 1 ? EHCI_SITD_TP_BEGIN : 0); // TP = Begin or All + } +#else + (void) dir; +#endif + // Reset status, split progress and page selection; retain endpoint/masks. + td->words[SITD_STATE_WORD] = ((uint32_t) ep->buflen << EHCI_SITD_STATE_BYTES_SHIFT) | EHCI_SITD_STATE_IOC; + } + // An unvisited slot raises no interrupt. Recheck just before linking: a + // delay since scheduling (preemption) moves the request to a later slot of + // its phase rather than publishing one the controller may have passed. + if ((int32_t) (ep->uframe - iso_earliest(iso_now())) >= 0) { + break; + } + } + // Publish an inactive descriptor and its link before publishing Active. + // The previous request in this slot was unlinked before its completion event. + ehci_link_t* head = &ehci_data.period_framelist[(ep->uframe >> 3) % FRAMELIST_SIZE]; + td->itd.next = *head; + TU_MEMORY_BARRIER(); + head->address = (uint32_t) td | ((ep->speed == TUSB_SPEED_HIGH ? EHCI_QTYPE_ITD : EHCI_QTYPE_SITD) << 1); + TU_MEMORY_BARRIER(); + if (ep->speed == TUSB_SPEED_HIGH) { + td->itd.xact[ep->uframe & 7].active = 1; + } else { + td->sitd.active = 1; + } + ep->armed = true; + TU_MEMORY_BARRIER(); +} + +static bool iso_xfer(uint8_t rhport, iso_ep_t* ep, uint8_t* buffer, uint16_t buflen) { + TU_VERIFY(!ep->busy && buflen <= ep->max_xfer_bytes); + TU_VERIFY(buffer != NULL || buflen == 0); + if (CFG_TUH_MEM_DCACHE_ENABLE && buflen != 0) { + if (tu_edpt_dir(ep->ep_addr) == TUSB_DIR_IN) { + TU_VERIFY(hcd_dcache_clean_invalidate(buffer, buflen)); + } else { + TU_VERIFY(hcd_dcache_clean(buffer, buflen)); + } + } + hcd_int_disable(rhport); + ep->buffer = buffer; + ep->buflen = buflen; + ep->armed = false; + ep->busy = true; + iso_arm(ep); + hcd_int_enable(rhport); + return true; +} + +static void iso_process(bool in_isr) { + (void) iso_now(); + for (size_t i = iso_ep_next(0); i < QTD_MAX; i = iso_ep_next(i + 1)) { + iso_ep_t* ep = &ehci_data.qtd_pool[i].iso; + if (!ep->busy) { + continue; + } + // A deferred TD is armed once its frame enters the ring; one whose slot + // passed meanwhile moves to a later slot of its phase. + if (!ep->armed) { + iso_arm(ep); + } + uint32_t const now = iso_now(); + // Future requests cannot complete yet. + if ((int32_t) (now - ep->uframe) < 0) { + continue; + } + bool active = true; + bool error = false; + uint32_t actual = 0; + iso_td_t* td = ep->armed ? ep->td : NULL; + if (ep->armed) { + if (ep->speed == TUSB_SPEED_HIGH) { + uint8_t const slot = ep->uframe & 7; + uint32_t const status = td->words[ITD_XACT_WORD(slot)]; + active = (status & EHCI_ITD_XACT_ACTIVE) != 0; + if (!active) { + error = (status & EHCI_ITD_XACT_ERRORS) != 0; + actual = tu_edpt_dir(ep->ep_addr) ? + (status >> EHCI_ITD_XACT_LENGTH_SHIFT) & EHCI_ITD_XACT_LENGTH_MASK : ep->buflen; + } + } else { + uint32_t const status = td->words[SITD_STATE_WORD]; + active = (status & EHCI_SITD_STATE_ACTIVE) != 0; + if (!active) { + error = (status & EHCI_SITD_STATE_ERRORS) != 0; + uint16_t const remaining = (status >> EHCI_SITD_STATE_BYTES_SHIFT) & EHCI_SITD_STATE_BYTES_MASK; + actual = ep->buflen - tu_min16(ep->buflen, remaining); + } + } + } + uint32_t const end = ep->speed == TUSB_SPEED_HIGH ? ep->uframe + 1 : + (ep->uframe & ~7u) + 8; + if (active && (int32_t) (now - end) < 0) { + continue; + } + if (active) { + // The service slot passed unvisited. Retire the TD once the controller + // cannot be on it (it may be revisiting the entry a ring later); until + // then a later interrupt retries. + if (ep->armed && !iso_td_idle(ep)) { + continue; + } + error = true; + } + // Once a frame-list entry has recurred, a cleared Active bit cannot prove + // which visit delivered the packet. Do not claim an on-time success. + if ((int32_t) (now - ((ep->uframe & ~7u) + FRAMELIST_SIZE * 8u)) >= 0) { + error = true; + } + if (error) { + actual = 0; // descriptor byte counts need not be valid on an error + } + TU_MEMORY_BARRIER(); + if (CFG_TUH_MEM_DCACHE_ENABLE && tu_edpt_dir(ep->ep_addr) && actual != 0) { + if (!hcd_dcache_invalidate(ep->buffer, actual)) { + error = true; + actual = 0; + } + } + if (ep->armed) { + iso_td_unlink(ep); + ep->armed = false; + } + ep->uframe += ep->interval; + ep->busy = false; + hcd_event_xfer_complete(ep->daddr, ep->ep_addr, actual, + error ? XFER_RESULT_FAILED : XFER_RESULT_SUCCESS, in_isr); + } +} + +// Wait until the controller cannot be on ep's TD, bounded to a few frames. +static bool iso_td_wait_idle(iso_ep_t const* ep) { + uint32_t const start = ehci_data.regs->frame_index; + for (uint32_t guard = 0; guard < 0x100000u; guard++) { + if (iso_td_idle(ep)) { + return true; + } + if (((ehci_data.regs->frame_index - start) & 0x3fffu) > 32u) { + return false; + } + } + return false; +} + +// Close unconditionally, or cancel only work whose service slot is still ahead +// and that the controller cannot be accessing. +static bool iso_stop(uint8_t rhport, iso_ep_t* ep, bool close) { + TU_VERIFY(close || ep->busy); + hcd_int_disable(rhport); + bool done = true; + if (ep->armed) { + if (close) { + // The buffer goes back unconditionally; stop periodic DMA only if the + // controller stays on the TD. + bool const stopped = !iso_td_wait_idle(ep) && ehci_disable_schedule(ehci_data.regs, true); + iso_td_unlink(ep); + if (stopped) { + ehci_enable_schedule(ehci_data.regs, true); + } + } else { + iso_td_t* td = ep->td; + bool const active = ep->speed == TUSB_SPEED_HIGH ? + td->itd.xact[ep->uframe & 7].active : td->sitd.active; + done = active && (int32_t) (ep->uframe - iso_earliest(iso_now())) >= 0 && iso_td_idle(ep); + if (done) { + iso_td_unlink(ep); + } + } + } + if (done) { + if (close) { + iso_ep_free(ep); + if (iso_ep_next(0) == QTD_MAX) { + ehci_data.regs->command_bm.int_threshold = ehci_data.iso_saved_itc; + } + } else { + // Keep the cancelled slot: the next submission advances from it in phase. + ep->armed = false; + ep->busy = false; + } + } + hcd_int_enable(rhport); + return done; +} +#endif + //--------------------------------------------------------------------+ // EHCI Interrupt Handler //--------------------------------------------------------------------+ @@ -601,11 +1243,16 @@ void async_advance_isr(uint8_t rhport) { (void) rhport; - ehci_qhd_t *qhd_pool = ehci_data.qhd_pool; for (uint32_t i = 0; i < QHD_MAX; i++) { - if (qhd_pool[i].removing) { - qhd_pool[i].removing = 0; - qhd_pool[i].used = 0; +#if EHCI_ISO_ENABLED + if (qhd_is_iso(i)) { + continue; + } +#endif + ehci_qhd_t* qhd = &ehci_data.qhd_pool[i].qhd; + if (qhd->removing) { + qhd->removing = 0; + qhd->used = 0; } } } @@ -627,13 +1274,23 @@ } // Check queue head for potential transfer complete (successful or error) -TU_ATTR_ALWAYS_INLINE static inline -void qhd_xfer_complete_isr(ehci_qhd_t * qhd) { - hcd_dcache_invalidate(qhd, sizeof(ehci_qhd_t)); // HC may have updated the overlay +static void qhd_xfer_complete_isr(ehci_qhd_t * qhd) { + // This pointer is software-owned; idle QHs need no hardware status reads. + ehci_qtd_t* const qtd = qhd->attached_qtd; + if (qtd == NULL) { + return; + } volatile ehci_qtd_t *qtd_overlay = &qhd->qtd_overlay; // process non-active (completed) QHD with attached (scheduled) TD - if ( !qtd_overlay->active && qhd->attached_qtd != NULL ) { + if ( !qtd_overlay->active ) { + // An unrelated periodic interrupt can arrive before the controller has + // fetched a newly attached qTD. The inactive overlay then still belongs + // to the previous transfer; only a retired qTD establishes completion. + if (qtd->active) { + return; + } + TU_MEMORY_BARRIER(); xfer_result_t xfer_result; if ( qtd_overlay->halted ) { @@ -652,9 +1309,6 @@ xfer_result = XFER_RESULT_SUCCESS; } - ehci_qtd_t * volatile qtd = qhd->attached_qtd; - hcd_dcache_invalidate(qtd, sizeof(ehci_qtd_t)); // HC may have written back TD - uint8_t const dir = (qtd->pid == EHCI_PID_IN) ? 1 : 0; uint32_t const xferred_bytes = qtd->expected_bytes - qtd->total_bytes; @@ -702,7 +1356,7 @@ } break; - // TODO support hs/fs ISO + // ISO descriptors are retired by iso_process(). case EHCI_QTYPE_ITD: case EHCI_QTYPE_SITD: case EHCI_QTYPE_FSTN: @@ -721,7 +1375,7 @@ uint32_t const int_status = regs->status; if (int_status & EHCI_INT_MASK_HC_HALTED) { - // something seriously wrong, maybe forget to flush/invalidate cache + // something seriously wrong, e.g. a descriptor placed in cacheable memory TU_BREAKPOINT(); TU_LOG1(" HC halted\r\n"); return; @@ -748,13 +1402,26 @@ // A USB transfer is completed (OK or error) uint32_t const usb_int = int_status & (EHCI_INT_MASK_USB | EHCI_INT_MASK_ERROR); if (usb_int) { + // Acknowledge before scanning: a completion that arrives after its QH was + // visited must remain pending for the next interrupt. + regs->status = usb_int; + } +#if EHCI_ISO_ENABLED + // Completions drive ISO. Frame-list rollover arms TDs beyond the current + // ring, retires slots that passed unvisited and samples FRINDEX well within + // its 14-bit wrap. + uint32_t const iso_tick = EHCI_INT_MASK_FRAMELIST_ROLLOVER; + if ((usb_int || (int_status & regs->inten & iso_tick)) && iso_ep_next(0) != QTD_MAX) { + iso_process(in_isr); + } +#endif + if (usb_int) { proccess_async_xfer_isr(list_get_async_head(rhport)); for ( uint32_t i = 1; i <= FRAMELIST_SIZE; i *= 2 ) { process_period_xfer_isr(rhport, i); } - regs->status = usb_int; // Acknowledge } //------------- There is some removed async previously -------------// @@ -787,7 +1454,9 @@ TU_ATTR_ALWAYS_INLINE static inline void list_insert(ehci_link_t *current, ehci_link_t *entry, uint8_t type) { entry->address = current->address; + TU_MEMORY_BARRIER(); current->address = ((uint32_t) entry) | (type << 1); + TU_MEMORY_BARRIER(); } // Remove a queue head from the list. @@ -796,6 +1465,7 @@ TU_ATTR_ALWAYS_INLINE static inline void list_remove(ehci_link_t* head, ehci_link_t* prev, ehci_qhd_t* qhd) { // TODO deactivate all TD, wait for QHD to inactive before removal prev->address = qhd->next.address; + TU_MEMORY_BARRIER(); // link the removed qhd's next to list head qhd->next.address = ((uint32_t) head) | (EHCI_QTYPE_QHD << 1); @@ -807,9 +1477,7 @@ // async list use async advance handshake. Mark as removing, will completely re-usable when async advance isr occurs qhd->removing = 1; } - - hcd_dcache_clean(qhd, sizeof(ehci_qhd_t)); - hcd_dcache_clean(prev, sizeof(ehci_qhd_t)); + TU_MEMORY_BARRIER(); } // Remove queue head belong to this device address @@ -844,10 +1512,15 @@ } // Find a free queue head -TU_ATTR_ALWAYS_INLINE static inline ehci_qhd_t *qhd_find_free(void) { +static ehci_qhd_t *qhd_find_free(void) { for (uint32_t i = 0; i < QHD_MAX; i++) { - if (!ehci_data.qhd_pool[i].used) { - return &ehci_data.qhd_pool[i]; +#if EHCI_ISO_ENABLED + if (qhd_is_iso(i)) { + continue; + } +#endif + if (!ehci_data.qhd_pool[i].qhd.used) { + return &ehci_data.qhd_pool[i].qhd; } } return NULL; @@ -858,27 +1531,39 @@ return (ehci_qhd_t *) tu_align32(p_qhd->next.address); } -// Get queue head from device + endpoint address -static ehci_qhd_t *qhd_get_from_addr(uint8_t dev_addr, uint8_t ep_addr) { - if ( 0 == tu_edpt_number(ep_addr) ) { - return qhd_control(dev_addr); +// Resolve either endpoint type once; control endpoints have dedicated QHs. +static ehci_edpt_t edpt_find(uint8_t dev_addr, uint8_t ep_addr) { + ehci_edpt_t result = {0}; + if (tu_edpt_number(ep_addr) == 0) { + result.qhd = qhd_control(dev_addr); + return result; } - ehci_qhd_t *qhd_pool = ehci_data.qhd_pool; - - // protect qhd_pool since 'used' and 'removing' can be changed in isr - ehci_qhd_t *result = NULL; +#if EHCI_ISO_ENABLED + // ISO lookup fields change only on task-context open/close; no IRQ lock needed. + for (size_t i = iso_ep_next(0); i < QTD_MAX; i = iso_ep_next(i + 1)) { + iso_ep_t* iso = &ehci_data.qtd_pool[i].iso; + if (iso->daddr == dev_addr && iso->ep_addr == ep_addr) { + result.iso = iso; + return result; + } + } +#endif + // Protect QH ownership flags, which can change during async advance. usbh_spin_lock(false); - for (uint32_t i = 0; i < QHD_MAX; i++) { - if ((qhd_pool[i].dev_addr == dev_addr) && - ep_addr == qhd_ep_addr(&qhd_pool[i]) && - qhd_pool[i].used && !qhd_pool[i].removing) { - result = &qhd_pool[i]; + for (size_t i = 0; i < QHD_MAX; i++) { +#if EHCI_ISO_ENABLED + if (qhd_is_iso(i)) { + continue; + } +#endif + ehci_qhd_t* qhd = &ehci_data.qhd_pool[i].qhd; + if (qhd->used && !qhd->removing && qhd->dev_addr == dev_addr && qhd_ep_addr(qhd) == ep_addr) { + result.qhd = qhd; break; } } usbh_spin_unlock(false); - return result; } @@ -932,10 +1617,6 @@ } break; - case TUSB_XFER_ISOCHRONOUS: - // TODO not support ISO yet - break; - default: break; } @@ -964,11 +1645,10 @@ qhd->attached_qtd = qtd; qhd->attached_buffer = qtd->buffer[0]; - // clean and invalidate cache before physically write - hcd_dcache_clean_invalidate(qtd, sizeof(ehci_qtd_t)); - + // Publish the initialized TD before linking it into the active queue. + TU_MEMORY_BARRIER(); qhd->qtd_overlay.next.address = (uint32_t) qtd; - hcd_dcache_clean_invalidate(qhd, sizeof(ehci_qhd_t)); + TU_MEMORY_BARRIER(); } // Remove an attached TD from queue head @@ -977,10 +1657,8 @@ qhd->attached_qtd = NULL; qhd->attached_buffer = 0; - hcd_dcache_clean(qhd, sizeof(ehci_qhd_t)); qtd->used = 0; // free QTD - hcd_dcache_clean(qtd, sizeof(ehci_qtd_t)); } //--------------------------------------------------------------------+ @@ -992,9 +1670,14 @@ return &ehci_data.control[dev_addr].qtd; } -TU_ATTR_ALWAYS_INLINE static inline ehci_qtd_t *qtd_find_free(void) { +static ehci_qtd_t *qtd_find_free(void) { for (uint32_t i = 0; i < QTD_MAX; i++) { - if (!ehci_data.qtd_pool[i].used) return &ehci_data.qtd_pool[i]; +#if EHCI_ISO_ENABLED + if (qtd_is_iso(i)) { + continue; + } +#endif + if (!ehci_data.qtd_pool[i].qtd.used) return &ehci_data.qtd_pool[i].qtd; } return NULL; }
diff --git a/src/portable/ehci/ehci.h b/src/portable/ehci/ehci.h index 9d86ed0..a2e23cc 100644 --- a/src/portable/ehci/ehci.h +++ b/src/portable/ehci/ehci.h
@@ -197,8 +197,8 @@ uint32_t direction : 1; ///< 0 = OUT; 1 = IN. This field encodes whether the full-speed transaction should be an IN or OUT. // Word 2: Micro-frame Schedule Control - uint8_t int_smask ; ///< This field (along with the Activeand SplitX-statefields in the Statusbyte) are used to determine during which micro-frames the host controller should execute complete-split transactions - uint8_t fl_int_cmask; ///< This field (along with the Activeand SplitX-statefields in the Statusbyte) are used to determine during which micro-frames the host controller should execute start-split transactions. + uint8_t int_smask ; ///< This field (along with the Activeand SplitX-statefields in the Statusbyte) are used to determine during which micro-frames the host controller should execute start-split transactions. + uint8_t fl_int_cmask; ///< This field (along with the Activeand SplitX-statefields in the Statusbyte) are used to determine during which micro-frames the host controller should execute complete-split transactions uint16_t reserved ; ///< reserved // Word 3: siTD Transfer Status and Control @@ -233,6 +233,29 @@ TU_VERIFY_STATIC( sizeof(ehci_sitd_t) == 32, "size is not correct" ); +// Raw views of the iTD transaction word (EHCI Table 3-3) and the siTD transfer +// state word (Table 3-11), for reading a controller-updated word only once. +#define EHCI_ITD_XACT_ACTIVE TU_BIT(31) +#define EHCI_ITD_XACT_ERRORS (TU_BIT(30) | TU_BIT(29) | TU_BIT(28)) // buffer, babble, transaction +#define EHCI_ITD_XACT_LENGTH_SHIFT 16 +#define EHCI_ITD_XACT_LENGTH_MASK 0xfffu +#define EHCI_ITD_XACT_IOC TU_BIT(15) + +#define EHCI_SITD_STATE_IOC TU_BIT(31) +#define EHCI_SITD_STATE_BYTES_SHIFT 16 +#define EHCI_SITD_STATE_BYTES_MASK 0x3ffu +#define EHCI_SITD_STATE_ACTIVE TU_BIT(7) +#define EHCI_SITD_STATE_ERRORS 0x7cu // error, buffer, babble, transaction, missed micro-frame + +// siTD endpoint characteristics (Table 3-9): device address in [6:0] +#define EHCI_SITD_EP_ENDPT_SHIFT 8 +#define EHCI_SITD_EP_HUB_SHIFT 16 +#define EHCI_SITD_EP_PORT_SHIFT 24 +#define EHCI_SITD_EP_DIR_SHIFT 31 + +// siTD buffer page 1 (Table 3-12): transaction count in [2:0], position in [4:3] +#define EHCI_SITD_TP_BEGIN TU_BIT(3) + //--------------------------------------------------------------------+ // EHCI Operational Register //--------------------------------------------------------------------+
diff --git a/src/tusb_option.h b/src/tusb_option.h index 659eb9f..6242a44 100644 --- a/src/tusb_option.h +++ b/src/tusb_option.h
@@ -823,6 +823,11 @@ #define CFG_TUH_TASK_EVENTS_PER_RUN 16 #endif +// Enable the ChipIdea HCD's single-transfer isochronous scheduler. +#ifndef CFG_TUH_CHIPIDEA_ISO_ENABLE + #define CFG_TUH_CHIPIDEA_ISO_ENABLE 0 +#endif + //------------- CLASS -------------// #ifndef CFG_TUH_HUB
diff --git a/test/unit-test/CMakeLists.txt b/test/unit-test/CMakeLists.txt index ff945d8..74bbf9d 100644 --- a/test/unit-test/CMakeLists.txt +++ b/test/unit-test/CMakeLists.txt
@@ -144,3 +144,4 @@ ) enable_testing() +add_subdirectory(host)
diff --git a/test/unit-test/host/CMakeLists.txt b/test/unit-test/host/CMakeLists.txt new file mode 100644 index 0000000..9934325 --- /dev/null +++ b/test/unit-test/host/CMakeLists.txt
@@ -0,0 +1,60 @@ +cmake_minimum_required(VERSION 3.20) + +# Also allow running the host fixtures without installing Ceedling. +if (CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR) + project(tinyusb_host_tests LANGUAGES C) + enable_testing() +endif () + +if (NOT CMAKE_C_COMPILER_ID MATCHES "^(GNU|Clang|AppleClang)$") + message(FATAL_ERROR "Host fixtures require GCC or Clang (MinGW on Windows).") +endif () + +get_filename_component(TINYUSB_ROOT "${CMAKE_CURRENT_LIST_DIR}/../../.." ABSOLUTE) + +function(add_host_test name fixture) + add_executable(${name} ${fixture}) + get_filename_component(fixture_dir "${fixture}" DIRECTORY) + target_include_directories(${name} PRIVATE ${fixture_dir} ${TINYUSB_ROOT}/src) + set_target_properties(${name} PROPERTIES C_STANDARD 99 C_STANDARD_REQUIRED YES) + # Keep assertions active even when configured with a release build type. + target_compile_options(${name} PRIVATE -O2 -g -Wall -Wextra -Werror -UNDEBUG) + add_test(NAME ${name} COMMAND ${name}) + set_tests_properties(${name} PROPERTIES TIMEOUT 10 LABELS host) +endfunction() + +# EHCI fixture links encode native pointers in 32 bits. macOS cannot guarantee +# low static addresses (in particular, Apple Silicon executables require PIE). +if (APPLE) + message(STATUS "Skipping EHCI fixtures on macOS: static DMA addresses must fit in 32 bits.") + return() +endif () + +foreach (variant IN ITEMS generic disabled enabled cached nohub) + set(name test_ehci_iso_${variant}) + add_host_test(${name} ${CMAKE_CURRENT_LIST_DIR}/ehci/test_iso.c) + target_sources(${name} PRIVATE ${CMAKE_CURRENT_LIST_DIR}/ehci/cache_stubs.c) + if (NOT variant STREQUAL "disabled") + target_compile_definitions(${name} PRIVATE CFG_TUH_CHIPIDEA_ISO_ENABLE=1) + endif () + if (variant STREQUAL "enabled" OR variant STREQUAL "cached") + target_sources(${name} PRIVATE ${TINYUSB_ROOT}/src/host/hub.c) + endif () + if (variant STREQUAL "nohub") + target_compile_definitions(${name} PRIVATE CFG_TUH_HUB=0) + endif () + if (variant STREQUAL "cached") + target_compile_definitions(${name} PRIVATE CFG_TUH_MEM_DCACHE_ENABLE=1) + endif () + if (variant STREQUAL "generic") + target_compile_definitions(${name} PRIVATE TEST_EHCI_GENERIC) + endif () + target_compile_options(${name} PRIVATE -Wno-pointer-to-int-cast -Wno-int-to-pointer-cast) + # EHCI hardware links are 32-bit; keep simulated DMA memory below 4 GiB. + if (WIN32) + target_link_options(${name} PRIVATE -Wl,--image-base,0x400000,--disable-dynamicbase) + else () + target_compile_options(${name} PRIVATE -fno-pie) + target_link_options(${name} PRIVATE -no-pie) + endif () +endforeach ()
diff --git a/test/unit-test/host/README.md b/test/unit-test/host/README.md new file mode 100644 index 0000000..664991c --- /dev/null +++ b/test/unit-test/host/README.md
@@ -0,0 +1,31 @@ +# Host driver tests + +These fixtures run through CTest and the host-test pre-commit hook: + +```sh +cmake -S test/unit-test/host -B build/host-test -DCMAKE_BUILD_TYPE=Debug +cmake --build build/host-test +ctest --test-dir build/host-test --output-on-failure +``` + +Use GCC or Clang (MinGW on Windows, with runtime DLLs on PATH). + +EHCI configurations cover generic EHCI, ChipIdea with ISO disabled/enabled, +and ChipIdea ISO with payload D-cache maintenance enabled. An additional configuration +runs with `CFG_TUH_HUB=0` to check direct HS/FS without hub dependencies. + +The enabled fixture uses the actual HCD with simulated DMA writeback and +FRINDEX. It checks native FS, split transfers, HS, single-request ownership, +shared QH/qTD allocation and exhaustion, early completion, TD reuse, errors, +cancellation, endpoint phase, and counter/frame-list wrap. Every-microframe +HS service is checked with exactly one TD per endpoint. + +The enabled fixture exercises the best-effort MTT split allocator, including +conflicting opens, separate hub ports, nested FS hubs, release on close, and +maximum-packet reservations retained across short and zero-length transfers. + +Assertions remain enabled in release builds; each test has a 10-second limit. +Static simulated DMA addresses are below 4 GiB. These tests do not emulate +DMA/cache coherency or real USB timing; hardware validation is still needed. +The EHCI fixtures are skipped on macOS, which cannot guarantee low static +addresses.
diff --git a/test/unit-test/host/ehci/cache_stubs.c b/test/unit-test/host/ehci/cache_stubs.c new file mode 100644 index 0000000..4a7f750 --- /dev/null +++ b/test/unit-test/host/ehci/cache_stubs.c
@@ -0,0 +1,97 @@ +// SPDX-License-Identifier: MIT +#include <stdbool.h> +#include <stdint.h> +#include <stdlib.h> +#include "tusb_option.h" +#include "host/usbh.h" +#include "host/usbh_pvt.h" + +unsigned test_cache_calls[3]; + +void test_cache_buffer(void const* addr, uint32_t size); + +// Reject cache maintenance on descriptor storage in every build. With cache +// disabled, retain USBH's false-returning default hooks used on LPC43. +bool hcd_dcache_clean(void const* addr, uint32_t data_size) { + test_cache_calls[0]++; + test_cache_buffer(addr, data_size); + return CFG_TUH_MEM_DCACHE_ENABLE != 0; +} + +bool hcd_dcache_invalidate(void const* addr, uint32_t data_size) { + test_cache_calls[1]++; + test_cache_buffer(addr, data_size); + return CFG_TUH_MEM_DCACHE_ENABLE != 0; +} + +bool hcd_dcache_clean_invalidate(void const* addr, uint32_t data_size) { + test_cache_calls[2]++; + test_cache_buffer(addr, data_size); + return CFG_TUH_MEM_DCACHE_ENABLE != 0; +} + +#if CFG_TUH_HUB && defined(TUP_USBIP_CHIPIDEA_HS) && CFG_TUH_CHIPIDEA_ISO_ENABLE +// Driver fixtures use the real hub allocator, but never enumerate or transfer +// through the hub class driver. Fail immediately if those paths are called. +bool tuh_descriptor_get_device_local(uint8_t daddr, tusb_desc_device_t* desc_device) { + (void) daddr; + (void) desc_device; + abort(); +} + +bool tuh_edpt_open(uint8_t daddr, tusb_desc_endpoint_t const* desc_ep) { + (void) daddr; + (void) desc_ep; + abort(); +} + +uint8_t usbh_get_rhport(uint8_t daddr) { + (void) daddr; + abort(); +} + +bool tuh_control_xfer(tuh_xfer_t* xfer) { + (void) xfer; + abort(); +} + +bool tuh_interface_set(uint8_t daddr, uint8_t itf_num, uint8_t itf_alt, + tuh_xfer_cb_t complete_cb, uintptr_t user_data) { + (void) daddr; + (void) itf_num; + (void) itf_alt; + (void) complete_cb; + (void) user_data; + abort(); +} + +bool usbh_edpt_claim(uint8_t dev_addr, uint8_t ep_addr) { + (void) dev_addr; + (void) ep_addr; + abort(); +} + +bool usbh_edpt_release(uint8_t dev_addr, uint8_t ep_addr) { + (void) dev_addr; + (void) ep_addr; + abort(); +} + +bool usbh_edpt_xfer_with_callback(uint8_t dev_addr, uint8_t ep_addr, uint8_t* buffer, uint16_t total_bytes, + tuh_xfer_cb_t complete_cb, uintptr_t user_data) { + (void) dev_addr; + (void) ep_addr; + (void) buffer; + (void) total_bytes; + (void) complete_cb; + (void) user_data; + abort(); +} + +void usbh_driver_set_config_complete(uint8_t dev_addr, uint8_t itf_num) { + (void) dev_addr; + (void) itf_num; + abort(); +} + +#endif
diff --git a/test/unit-test/host/ehci/test_iso.c b/test/unit-test/host/ehci/test_iso.c new file mode 100644 index 0000000..83a4f43 --- /dev/null +++ b/test/unit-test/host/ehci/test_iso.c
@@ -0,0 +1,1154 @@ +// SPDX-License-Identifier: MIT +#include <assert.h> +#include <stdio.h> +#include "tusb_option.h" +#include "common/tusb_common.h" + +// The QH software tail and ISO state contain native pointers. On a 64-bit test +// host their sizes differ from the target ABI. Recheck hardware layouts below. +#pragma push_macro("TU_VERIFY_STATIC") +#undef TU_VERIFY_STATIC +#define TU_VERIFY_STATIC(condition, ...) \ + enum { TU_XSTRCAT(_verify_static_, TU_COUNTER) = 1 / (!!((condition) || sizeof(void*) == 8)) } +#include "portable/ehci/ehci.h" +#include "portable/ehci/ehci.c" +#pragma pop_macro("TU_VERIFY_STATIC") + +TU_VERIFY_STATIC(sizeof(ehci_link_t) == 4, "link ABI"); +TU_VERIFY_STATIC(sizeof(ehci_qtd_t) == 32, "qTD ABI"); +TU_VERIFY_STATIC(offsetof(ehci_qhd_t, qtd_overlay) == 16, "QH hardware prefix"); +TU_VERIFY_STATIC(sizeof(ehci_itd_t) == 64, "iTD ABI"); +TU_VERIFY_STATIC(sizeof(ehci_sitd_t) == 32, "siTD ABI"); +TU_VERIFY_STATIC(sizeof(ehci_cap_registers_t) == 16, "capability register ABI"); + +extern unsigned test_cache_calls[3]; + +// Cache hooks may touch application buffers, never uncached controller storage. +void test_cache_buffer(void const* addr, uint32_t size) { + uintptr_t const start = (uintptr_t) addr; + uintptr_t const descriptors = (uintptr_t) &ehci_data; + assert(start + size <= descriptors || start >= descriptors + sizeof(ehci_data)); +} + +static ehci_registers_t regs; +static ehci_cap_registers_t caps; +static tuh_bus_info_t buses[8]; +static hcd_event_t event; +static unsigned events; +static hcd_event_t terminal_events[16]; +static uint8_t buffer[8192] TU_ATTR_ALIGNED(4096); + +void hcd_int_enable(uint8_t rhport) { (void) rhport; } +void hcd_int_disable(uint8_t rhport) { (void) rhport; } +void usbh_spin_lock(bool in_isr) { (void) in_isr; } +void usbh_spin_unlock(bool in_isr) { (void) in_isr; } +bool tuh_bus_info_get(uint8_t daddr, tuh_bus_info_t* bus) { + memset(bus, 0, sizeof(*bus)); + if (daddr >= TU_ARRAY_SIZE(buses)) { + return false; + } + *bus = buses[daddr]; + return true; +} +void hcd_event_handler(hcd_event_t const* e, bool in_isr) { + (void) in_isr; + event = *e; + assert(events < TU_ARRAY_SIZE(terminal_events)); + terminal_events[events++] = *e; +} + +static void reset(uint8_t root_speed) { + memset(&ehci_data, 0, sizeof(ehci_data)); + memset((void*)®s, 0, sizeof(regs)); + memset((void*)&caps, 0, sizeof(caps)); + memset(buses, 0, sizeof(buses)); + ehci_data.regs = ®s; + ehci_data.cap_regs = ∩︀ + regs.portsc = (uint32_t)root_speed << 26; + regs.frame_index = 800; + regs.command_bm.int_threshold = 8; + init_periodic_list(0); + events = 0; +} + +static void test_attach_debounce(void) { + reset(TUSB_SPEED_FULL); + regs.portsc |= EHCI_PORTSC_MASK_CURRENT_CONNECT_STATUS; + uint32_t const before = regs.portsc; + port_connect_status_change_isr(0); + assert(events == 1 && event.event_id == HCD_EVENT_DEVICE_ATTACH); + assert(regs.portsc == before); // Attach must not reset the port before USBH debounces it. +} + +static void test_endpoint_lookup(void) { + reset(TUSB_SPEED_HIGH); + // Same endpoint number in both directions, and the same address on another device. + uint8_t const dev[] = {1, 1, 2}; + uint8_t const addr[] = {1, 0x81, 0x81}; + ehci_qhd_t* qhd[3]; + for (unsigned i = 0; i < TU_ARRAY_SIZE(dev); i++) { + buses[dev[i]].speed = TUSB_SPEED_HIGH; + tusb_desc_endpoint_t const desc = { + .bLength = sizeof(desc), .bDescriptorType = TUSB_DESC_ENDPOINT, + .bEndpointAddress = addr[i], .bmAttributes = {.xfer = TUSB_XFER_INTERRUPT}, + .wMaxPacketSize = 64, .bInterval = 1 + }; + assert(hcd_edpt_open(0, dev[i], &desc)); + qhd[i] = &ehci_data.qhd_pool[i].qhd; + ehci_edpt_t const found = edpt_find(dev[i], addr[i]); + assert(found.qhd == qhd[i]); +#if EHCI_ISO_ENABLED + assert(found.iso == NULL); +#endif + } + for (unsigned i = 0; i < TU_ARRAY_SIZE(ehci_data.control); i++) { + assert(edpt_find(i, 0).qhd == qhd_control(i)); + assert(edpt_find(i, 0x80).qhd == qhd_control(i)); + } + assert(edpt_find(1, 0x82).qhd == NULL); + qhd[1]->removing = 1; + assert(edpt_find(1, 0x81).qhd == NULL); + qhd[1]->removing = 0; + qhd[1]->used = 0; + assert(edpt_find(1, 0x81).qhd == NULL); // A stale key cannot match a free slot. + assert(edpt_find(1, 1).qhd == qhd[0]); + assert(edpt_find(2, 0x81).qhd == qhd[2]); +} + +#if EHCI_ISO_ENABLED +#define TEST_ISO_STREAM_EP_COUNT 4 + +static iso_ep_t* iso_ep_find(uint8_t daddr, uint8_t ep_addr) { + return edpt_find(daddr, ep_addr).iso; +} + +static ehci_qhd_t* qhd_get_from_addr(uint8_t daddr, uint8_t ep_addr) { + return edpt_find(daddr, ep_addr).qhd; +} + +static bool open_ep(uint8_t addr, uint8_t speed, uint16_t size, uint8_t interval) { + buses[1].speed = speed; + tusb_desc_endpoint_t desc = { + .bLength = sizeof(desc), .bDescriptorType = TUSB_DESC_ENDPOINT, + .bEndpointAddress = addr, .bmAttributes = {.xfer = TUSB_XFER_ISOCHRONOUS}, + .wMaxPacketSize = size, .bInterval = interval + }; + return iso_ep_open(0, 1, &desc); +} + +// Move the simulated controller clock so iso_now() returns now. +static void set_now(uint32_t now) { + ehci_data.iso_uframe = now + ehci_data.iso_frame_offset; + ehci_data.iso_last_frindex = (uint16_t) (ehci_data.iso_uframe & 0x3fff); + regs.frame_index = ehci_data.iso_last_frindex; +} + +static void test_shared_pools(void) { + reset(TUSB_SPEED_HIGH); + buses[1].speed = TUSB_SPEED_HIGH; + tusb_desc_endpoint_t const intr_desc = { + .bLength = sizeof(intr_desc), .bDescriptorType = TUSB_DESC_ENDPOINT, + .bEndpointAddress = 0x83, .bmAttributes = {.xfer = TUSB_XFER_INTERRUPT}, + .wMaxPacketSize = 64, .bInterval = 1 + }; + assert(hcd_edpt_open(0, 1, &intr_desc)); + assert(hcd_edpt_xfer(0, 1, 0x83, buffer, 64)); + ehci_qhd_t* qhd = qhd_get_from_addr(1, 0x83); + ehci_qtd_t* qtd = qhd->attached_qtd; + assert(qtd == &ehci_data.qtd_pool[0].qtd); + + // ISO uses one free QH for its TD and one free qTD for its state. + assert(open_ep(0x81, TUSB_SPEED_HIGH, 64, 1)); + iso_ep_t* ep = iso_ep_find(1, 0x81); + assert(qhd_get_from_addr(1, 0x81) == NULL); + assert(qtd_find_free() == &ehci_data.qtd_pool[2].qtd); + assert(ep->td == &ehci_data.qhd_pool[1].iso); + assert(((uintptr_t) ep->td & 63) == 0); + // Async reclamation must not interpret an ISO descriptor as a QH. + iso_ep_t const saved = *ep; + async_advance_isr(0); + assert(memcmp(ep, &saved, sizeof(saved)) == 0); + assert(qhd_get_from_addr(1, 0x83) == qhd); + + // Release unpublished ISO descriptors and reuse their space as qTDs. + iso_ep_free(ep); + assert(iso_ep_find(1, 0x81) == NULL); + ehci_qtd_t* second = qtd_find_free(); + qtd_init(second, buffer + 64, 32); + ehci_qtd_t* recycled = qtd_find_free(); + assert(recycled == &ehci_data.qtd_pool[2].qtd); + qtd_init(recycled, buffer + 128, 16); + assert(open_ep(0x81, TUSB_SPEED_HIGH, 64, 1)); + assert(iso_ep_find(1, 0x81) == &ehci_data.qtd_pool[3].iso); + assert(iso_ep_find(1, 0x81)->td == &ehci_data.qhd_pool[1].iso); + assert(second->active && recycled->active && qtd->active); + qtd->active = 0; + qtd->total_bytes = 0; + qhd_xfer_complete_isr(qhd); + assert(events == 1 && event.xfer_complete.len == 64 && !qtd->used); + assert(second->active && recycled->active); + + // Exhaust the TD pool and check that failed open releases its QH slot. + reset(TUSB_SPEED_HIGH); + for (size_t i = 0; i < QTD_MAX; i++) { + qtd = qtd_find_free(); + assert(qtd != NULL); + qtd_init(qtd, buffer, 64); + } + assert(qtd_find_free() == NULL); + assert(!open_ep(0x81, TUSB_SPEED_HIGH, 64, 1)); + assert(iso_ep_find(1, 0x81) == NULL); + assert(qhd_find_free() == &ehci_data.qhd_pool[0].qhd); + for (size_t i = 0; i < TU_ARRAY_SIZE(ehci_data.qtd_is_iso); i++) { + assert(!ehci_data.qtd_is_iso[i]); + } + for (size_t i = 0; i < QTD_MAX; i++) { + qtd = &ehci_data.qtd_pool[i].qtd; + assert(qtd->used && qtd->active && qtd->expected_bytes == 64); + } +} + +static void test_native_fs(void) { + reset(TUSB_SPEED_FULL); + assert(open_ep(0x81, TUSB_SPEED_FULL, 1023, 1)); + iso_ep_t* ep = iso_ep_find(1, 0x81); + assert(iso_xfer(0, ep, buffer + 4090, 1023)); + ehci_sitd_t* td = &ep->td->sitd; + assert(ep->uframe == 800); + assert(td->active && td->int_on_complete && td->total_bytes == 1023); + assert(td->int_smask == 0 && td->fl_int_cmask == 0); + assert(td->buffer[0] == (uint32_t)(uintptr_t)(buffer + 4090)); + assert(td->buffer[1] == (uint32_t)(uintptr_t)(buffer + 4096)); + + td->active = 0; + td->total_bytes = 23; + regs.frame_index = 808; + iso_process(true); + assert(events == 1 && event.xfer_complete.len == 1000); + assert(event.xfer_complete.result == XFER_RESULT_SUCCESS); + iso_process(true); + assert(events == 1); + assert(iso_xfer(0, ep, buffer, 0)); + td = &ep->td->sitd; + td->active = 0; + regs.frame_index += 8; + iso_process(true); + assert(events == 2 && event.xfer_complete.len == 0); + assert(event.xfer_complete.result == XFER_RESULT_SUCCESS); +} + +static void test_sparse_iso_pool(void) { + reset(TUSB_SPEED_HIGH); + // Keep the first 31 QHs/qTDs occupied by other endpoint types. ISO lookup and + // completion must handle both sides of the ownership bitmap boundary. + TU_VERIFY_STATIC(QHD_MAX > 32, "fixture must exercise multiple bitmap words"); + for (size_t i = 0; i < 31; i++) { + ehci_data.qhd_pool[i].qhd.used = 1; + ehci_data.qtd_pool[i].qtd.used = 1; + } + assert(open_ep(0x81, TUSB_SPEED_HIGH, 64, 1)); + assert(open_ep(0x82, TUSB_SPEED_HIGH, 64, 1)); + iso_ep_t* first = iso_ep_find(1, 0x81); + iso_ep_t* second = iso_ep_find(1, 0x82); + assert(first == &ehci_data.qtd_pool[31].iso); + assert(second == &ehci_data.qtd_pool[32].iso); + assert(iso_xfer(0, first, buffer, 64)); + assert(iso_xfer(0, second, buffer + 64, 64)); + first->td->itd.xact[first->uframe & 7].active = 0; + second->td->itd.xact[second->uframe & 7].active = 0; + regs.frame_index = first->uframe; + iso_process(true); + assert(events == 2 && !first->busy && !second->busy); + assert(terminal_events[0].xfer_complete.ep_addr == 0x81); + assert(terminal_events[1].xfer_complete.ep_addr == 0x82); + iso_ep_free(first); + assert(iso_ep_find(1, 0x81) == NULL); + assert(iso_ep_find(1, 0x82) == second); + assert(qhd_find_free() == &ehci_data.qhd_pool[31].qhd); + iso_ep_free(second); + assert(iso_ep_next(0) == QHD_MAX); +} + +#if CFG_TUH_HUB +static void test_split(void) { + reset(TUSB_SPEED_HIGH); + buses[1].hub_addr = 2; + buses[1].hub_port = 3; + buses[2].speed = TUSB_SPEED_FULL; + buses[2].hub_addr = 3; + buses[2].hub_port = 4; + buses[3].speed = TUSB_SPEED_HIGH; + assert(!open_ep(0x81, TUSB_SPEED_FULL, 565, 1)); + assert(open_ep(0x81, TUSB_SPEED_FULL, 564, 1)); + iso_ep_t* ep = iso_ep_find(1, 0x81); + assert(iso_xfer(0, ep, buffer, 564)); + ehci_sitd_t* td = &ep->td->sitd; + assert(td->hub_addr == 3 && td->port_number == 4); + assert(td->int_smask == 2 && td->fl_int_cmask == 0xf8); + assert(!open_ep(1, TUSB_SPEED_FULL, 1023, 1)); + assert(iso_ep_find(1, 1) == NULL); + // A different upstream MTT port has an independent budget. + buses[2].hub_port = 5; + assert(open_ep(1, TUSB_SPEED_FULL, 1023, 1)); + ep = iso_ep_find(1, 1); + assert(iso_xfer(0, ep, buffer, 1023)); + td = &ep->td->sitd; + assert(td->int_smask == 0x3f && td->fl_int_cmask == 0); + assert((td->buffer[1] & 0xfff) == (6 | 8)); +} + +static void test_split_audio(void) { + reset(TUSB_SPEED_HIGH); + buses[1].hub_addr = 2; + buses[1].hub_port = 1; + buses[2].speed = TUSB_SPEED_HIGH; + assert(open_ep(0x81, TUSB_SPEED_FULL, 98, 1)); + assert(open_ep(0x01, TUSB_SPEED_FULL, 196, 1)); + iso_ep_t* in = iso_ep_find(1, 0x81); + iso_ep_t* out = iso_ep_find(1, 0x01); + assert(iso_xfer(0, in, buffer, 98)); + assert(iso_xfer(0, out, buffer + 128, 192)); + ehci_sitd_t* in_td = &in->td->sitd; + ehci_sitd_t* out_td = &out->td->sitd; + // 48 kHz stereo needs two start-splits. Do not interleave an IN start + // with the OUT Begin/End sequence on the same transaction translator. + assert(out_td->int_smask == 3); + assert((out_td->buffer[1] & 0x1f) == (2 | 8)); + assert((in_td->int_smask & out_td->int_smask) == 0); + assert(in_td->int_smask > out_td->int_smask); +} + +static void test_split_allocation(void) { + // Both open orders support three OUT slots followed by one IN slot. + for (unsigned in_first = 0; in_first < 2; in_first++) { + reset(TUSB_SPEED_HIGH); + buses[1].hub_addr = 2; + buses[1].hub_port = 1; + buses[2].speed = TUSB_SPEED_HIGH; + if (in_first) { assert(open_ep(0x81, TUSB_SPEED_FULL, 100, 1)); } + assert(open_ep(0x01, TUSB_SPEED_FULL, 384, 1)); + if (!in_first) { assert(open_ep(0x81, TUSB_SPEED_FULL, 100, 1)); } + iso_ep_t* in = iso_ep_find(1, 0x81); + iso_ep_t* out = iso_ep_find(1, 0x01); + assert(out->split_slot == 0 && in->split_slot == 3); + assert(in->td->sitd.int_smask == 8 && in->td->sitd.fl_int_cmask == 0xe0); + assert(!open_ep(0x82, TUSB_SPEED_FULL, 4, 1)); + assert(iso_ep_find(1, 0x82) == NULL); + // Closing an idle endpoint releases its reservation. + iso_ep_free(out); + assert(open_ep(0x82, TUSB_SPEED_FULL, 4, 1)); + assert(iso_ep_find(1, 0x82)->split_slot == 2); + } + + reset(TUSB_SPEED_HIGH); + buses[1].hub_addr = 2; + buses[1].hub_port = 1; + buses[2].speed = TUSB_SPEED_HIGH; + assert(open_ep(0x01, TUSB_SPEED_FULL, 192, 1)); + assert(open_ep(0x81, TUSB_SPEED_FULL, 100, 1)); + assert(open_ep(0x82, TUSB_SPEED_FULL, 4, 1)); + assert(iso_ep_find(1, 0x82)->split_slot == 2); + assert(open_ep(0x02, TUSB_SPEED_FULL, 376, 1)); + iso_ep_t* out = iso_ep_find(1, 0x02); + assert(out->split_slot == 4); + uint16_t const lengths[] = {376, 189, 188, 1, 0}; + uint8_t const masks[] = {0x30, 0x30, 0x10, 0x10, 0x10}; + for (unsigned i = 0; i < TU_ARRAY_SIZE(lengths); i++) { + assert(iso_xfer(0, out, buffer, lengths[i])); + assert(out->td->sitd.int_smask == masks[i]); + // Short/ZLP requests do not release the maximum-packet reservation. + assert(!open_ep(0x03, TUSB_SPEED_FULL, 1, 1)); + iso_td_unlink(out); + out->armed = false; + out->busy = false; + } + buses[1].hub_port = 2; + assert(open_ep(0x03, TUSB_SPEED_FULL, 1023, 1)); + assert(iso_ep_find(1, 0x03)->split_slot == 0); + buses[1].hub_addr = 3; + buses[1].hub_port = 1; + buses[3].speed = TUSB_SPEED_HIGH; + assert(open_ep(0x04, TUSB_SPEED_FULL, 1023, 1)); + assert(iso_ep_find(1, 0x04)->split_slot == 0); +} + +#endif + +static void test_hs(void) { + reset(TUSB_SPEED_HIGH); + assert(open_ep(0x82, TUSB_SPEED_HIGH, 1024 | (2 << 11), 1)); + iso_ep_t* ep = iso_ep_find(1, 0x82); + assert(!iso_xfer(0, ep, buffer, 3073)); + assert(iso_xfer(0, ep, buffer + 4095, 3072)); + assert(!iso_xfer(0, ep, buffer, 64)); + assert(ep->buffer == buffer + 4095); + ehci_itd_t* td = &ep->td->itd; + uint8_t slot = ep->uframe & 7; + assert(ep->uframe == 801); + assert(((uintptr_t)td & 63) == 0); + assert(td->xact[slot].offset == 4095 && td->xact[slot].length == 3072); + assert((td->BufferPointer[0] & 0xfff) == 0x201); + assert((td->BufferPointer[1] & 0xfff) == 0xc00); + assert((td->BufferPointer[2] & 0xfff) == 3); + assert((td->BufferPointer[0] & ~0xfffu) == (uint32_t)(uintptr_t)buffer); + assert((td->BufferPointer[1] & ~0xfffu) == (uint32_t)(uintptr_t)(buffer + 4096)); + // Even the largest transfer at the last byte of a page only needs two pages. + assert(td->BufferPointer[2] == 3); + for (unsigned i = 0; i < 8; i++) { + assert(td->xact[i].active == (i == slot)); + } + td->xact[slot].active = 0; + td->xact[slot].length = 2048; + regs.frame_index = 803; + iso_process(true); + assert(events == 1 && event.xfer_complete.len == 2048); + assert(iso_xfer(0, ep, buffer, 10)); + td = &ep->td->itd; + slot = ep->uframe & 7; + td->xact[slot].active = 0; + td->xact[slot].babble_err = 1; + regs.frame_index = ep->uframe + 1; + iso_process(true); + assert(events == 2 && event.xfer_complete.result == XFER_RESULT_FAILED); + assert(event.xfer_complete.len == 0); +} + +static void test_long_interval_and_wrap(void) { + reset(TUSB_SPEED_HIGH); + assert(open_ep(1, TUSB_SPEED_HIGH, 64, 10)); + iso_ep_t* ep = iso_ep_find(1, 1); + assert(iso_xfer(0, ep, buffer, 64)); + assert(!ep->armed && ep->uframe == 1312); + regs.frame_index = 1264; + iso_process(true); + assert(ep->armed && events == 0); + ep->td->itd.xact[0].active = 0; + regs.frame_index = 1313; + iso_process(true); + assert(events == 1 && event.xfer_complete.len == 64); + assert(iso_xfer(0, ep, buffer, 64)); + assert(!ep->armed && iso_stop(0, ep, false)); + assert(!ep->busy && events == 1); + // Start a separate long interval request and let its slot pass unarmed: it + // moves to the next slot of its phase instead of failing. + ep->uframe = regs.frame_index + ep->interval; + assert(iso_xfer(0, ep, buffer, 64)); + uint32_t const missed = ep->uframe; + regs.frame_index = ep->uframe + 1; + iso_process(true); + assert(ep->busy && !ep->armed && events == 1 && ep->uframe == missed + ep->interval); + assert(iso_stop(0, ep, false) && !ep->busy && events == 1); + ehci_data.iso_last_frindex = 16380; + ehci_data.iso_uframe = 0xfffffffcu; + regs.frame_index = 4; + assert(iso_now() == 4); +} + +static void test_iso_status_errors(void) { + for (unsigned hs = 0; hs < 2; hs++) { + for (unsigned err = 0; err < (hs ? 3u : 5u); err++) { + reset(hs ? TUSB_SPEED_HIGH : TUSB_SPEED_FULL); + assert(open_ep(0x81, hs ? TUSB_SPEED_HIGH : TUSB_SPEED_FULL, 64, 1)); + iso_ep_t* ep = iso_ep_find(1, 0x81); + assert(iso_xfer(0, ep, buffer, 64)); + iso_td_t* td = ep->td; + // Active work must remain queued, even if status bits are already set. + if (hs) { + unsigned const slot = ep->uframe & 7; + switch (err) { + case 0: td->itd.xact[slot].error = 1; break; + case 1: td->itd.xact[slot].babble_err = 1; break; + case 2: td->itd.xact[slot].buffer_err = 1; break; + } + iso_process(true); + assert(events == 0); + td->itd.xact[slot].active = 0; + } else { + switch (err) { + case 0: td->sitd.error = 1; break; + case 1: td->sitd.buffer_err = 1; break; + case 2: td->sitd.babble_err = 1; break; + case 3: td->sitd.xact_err = 1; break; + case 4: td->sitd.missed_uframe = 1; break; + } + iso_process(true); + assert(events == 0); + td->sitd.active = 0; + } + regs.frame_index = (ep->uframe + ehci_data.iso_frame_offset) & 0x3fff; + iso_process(true); + assert(events == 1 && event.xfer_complete.result == XFER_RESULT_FAILED); + assert(event.xfer_complete.len == 0); + } + } +} + +static void test_iso_clock_config(void) { + reset(TUSB_SPEED_HIGH); + assert(open_ep(0x81, TUSB_SPEED_HIGH, 64, 1)); + assert(iso_now() == 800 && iso_earliest(iso_now()) == 801); + // Model completed teardown; this fixture cannot emulate the DMA stop/start handshake. + iso_ep_free(iso_ep_find(1, 0x81)); + init_periodic_list(0); + // A new root connection must refresh the native-FS frame offset. + regs.portsc = (uint32_t) TUSB_SPEED_FULL << 26; + assert(open_ep(0x81, TUSB_SPEED_FULL, 64, 1)); + assert(iso_now() == 792 && iso_earliest(iso_now()) == 793); +} + +#endif + +static void test_qtd_retirement(void) { + reset(TUSB_SPEED_FULL); + ehci_qhd_t* qh = &ehci_data.control[1].qhd; + ehci_qtd_t* td = &ehci_data.control[1].qtd; + qh->dev_addr = 1; + qh->attached_qtd = td; + td->active = 1; + td->expected_bytes = 3; + qhd_xfer_complete_isr(qh); + assert(events == 0 && qh->attached_qtd == td); + td->active = 0; + qhd_xfer_complete_isr(qh); + assert(events == 1 && event.xfer_complete.len == 3); + qhd_xfer_complete_isr(qh); + assert(events == 1); +} + +#if EHCI_ISO_ENABLED +static void test_limits_and_late_completion(void) { + reset(TUSB_SPEED_HIGH); + assert(!open_ep(0x80, TUSB_SPEED_HIGH, 64, 1)); + assert(!open_ep(0x81, TUSB_SPEED_LOW, 64, 1)); + assert(!open_ep(0x81, TUSB_SPEED_HIGH, 64, 0)); + assert(!open_ep(0x81, TUSB_SPEED_HIGH, 64, 17)); + assert(!open_ep(0x81, TUSB_SPEED_HIGH, 0, 1)); + assert(!open_ep(0x81, TUSB_SPEED_HIGH, 1025, 1)); + assert(!open_ep(0x81, TUSB_SPEED_HIGH, 64 | (3 << 11), 1)); + // Every slot, including the odd final slot, can serve an ISO endpoint. + for (unsigned i = 0; i <= QTD_MAX; i++) { + uint8_t const daddr = (uint8_t)(i / 30 + 1); + buses[daddr].speed = TUSB_SPEED_HIGH; + tusb_desc_endpoint_t const desc = { + .bLength = sizeof(desc), .bDescriptorType = TUSB_DESC_ENDPOINT, + .bEndpointAddress = tu_edpt_addr(i % 15 + 1, (i % 30) / 15), + .bmAttributes = {.xfer = TUSB_XFER_ISOCHRONOUS}, + .wMaxPacketSize = 64, .bInterval = 1 + }; + assert(iso_ep_open(0, daddr, &desc) == (i < QTD_MAX)); + if (i < QTD_MAX) { + iso_td_t* td = iso_ep_find(daddr, desc.bEndpointAddress)->td; + assert(((uintptr_t)td & 63) == 0); // Every iTD stays within a 4 KiB page. + } + } + assert(qhd_find_free() == NULL && qtd_find_free() == NULL); + iso_ep_t* ep = iso_ep_find(1, 1); + assert(iso_xfer(0, ep, buffer, 64)); + ep->td->itd.xact[ep->uframe & 7].active = 0; + regs.frame_index = (ep->uframe & ~7u) + FRAMELIST_SIZE * 8u; + iso_process(true); + assert(events == 1 && event.xfer_complete.result == XFER_RESULT_FAILED); + assert(!ep->busy); + for (size_t i = 0; i < QTD_MAX; i++) { + iso_ep_free(&ehci_data.qtd_pool[i].iso); + } + assert(iso_ep_next(0) == QTD_MAX); + assert(qhd_find_free() == &ehci_data.qhd_pool[0].qhd); + assert(qtd_find_free() == &ehci_data.qtd_pool[0].qtd); +} + + +static void test_schedule_sweep(void) { + // Exercise every descriptor slot, all intervals and frame-counter wrap. + for (uint8_t interval = 1; interval <= 16; interval++) { + for (uint32_t start = 16368; start < 16400; start++) { + reset(TUSB_SPEED_HIGH); + regs.frame_index = start & 0x3fff; + assert(open_ep(0x81, TUSB_SPEED_HIGH, 64, interval)); + iso_ep_t* ep = iso_ep_find(1, 0x81); + assert(iso_xfer(0, ep, buffer, 64)); + uint32_t const due = ep->uframe; + uint32_t const now = iso_now(); + assert((int32_t)(due - now) >= 1); + assert((due - now) % ep->interval == 0); + assert(ep->armed == (due - now < (FRAMELIST_SIZE - 1) * 8)); + if (!ep->armed) { + assert(iso_stop(0, ep, false)); + } else { + ep->td->itd.xact[due & 7].active = 0; + regs.frame_index = (due + 1) & 0x3fff; + iso_process(true); + assert(events == 1 && event.xfer_complete.result == XFER_RESULT_SUCCESS); + } + } + } +} + +static void test_descriptor_reuse(void) { + // Revisit the same descriptor with new lengths/pages and (for HS) microframes. + for (unsigned mode = 0; mode < (CFG_TUH_HUB ? 3u : 2u); mode++) { + for (unsigned dir = 0; dir < 2; dir++) { + bool const hs = mode == 0; + reset(mode == 1 ? TUSB_SPEED_FULL : TUSB_SPEED_HIGH); + if (mode == 2) { + buses[1].hub_addr = 2; + buses[1].hub_port = 3; + buses[2].speed = TUSB_SPEED_HIGH; + } + uint8_t const addr = tu_edpt_addr(1, dir); + uint16_t const mps = hs ? 1024 : (dir ? 564 : 1023); + assert(open_ep(addr, hs ? TUSB_SPEED_HIGH : TUSB_SPEED_FULL, + hs ? mps | (2 << 11) : mps, 1)); + iso_ep_t* ep = iso_ep_find(1, addr); + for (unsigned round = 0; round < 16; round++) { + ep->uframe = round * FRAMELIST_SIZE * 8 + (hs ? round & 7 : 0); + ep->buffer = buffer + ((round & 1) ? 0 : 4095); + ep->buflen = round % 3 == 0 ? mps * (hs ? 3 : 1) : (round % 3 == 1 ? 192 : 0); + ep->armed = false; + set_now(ep->uframe - 2); + iso_arm(ep); + assert(ep->armed); + iso_td_t* td = ep->td; + uint32_t const ptr = (uint32_t)(uintptr_t)ep->buffer; + uint32_t const page = ptr & ~0xfffu; + if (hs) { + unsigned const slot = ep->uframe & 7; + for (unsigned i = 0; i < 8; i++) { + assert(td->itd.xact[i].active == (i == slot)); + if (i != slot) { assert(td->words[1 + i] == 0); } + } + assert(td->itd.xact[slot].length == ep->buflen); + assert(td->itd.xact[slot].offset == (ptr & 0xfff)); + assert(td->itd.xact[slot].page_select == 0 && td->itd.xact[slot].int_on_complete); + assert(td->itd.BufferPointer[0] == (page | 0x101)); + assert(td->itd.BufferPointer[1] == ((page + 4096) | mps | (dir << 11))); + assert(td->itd.BufferPointer[2] == 3); + // Retired hardware status, including changed page/offset and errors. + td->words[1 + slot] = 0x7fffffff; + } else { + ehci_sitd_t* s = &td->sitd; + assert(s->dev_addr == 1 && s->ep_number == 1 && s->direction == dir); + assert(s->hub_addr == (mode == 2 ? 2 : 0)); + assert(s->port_number == (mode == 2 ? 3 : 0)); + assert(s->back.terminate && s->active && s->int_on_complete); + assert(s->total_bytes == ep->buflen && !s->cmask_progress && !s->page_select); + assert(!s->split_state && !s->missed_uframe && !s->xact_err && !s->error); + assert(!s->buffer_err && !s->babble_err && s->buffer[0] == ptr); + unsigned const count = ep->buflen ? (ep->buflen + 187) / 188 : 1; + assert(s->int_smask == (mode == 1 ? 0 : (dir ? 2 : (1u << count) - 1))); + assert(s->fl_int_cmask == (mode == 2 && dir ? 0xf8 : 0)); + assert(s->buffer[1] == ((page + 4096) | + (mode == 2 && !dir ? count | (count > 1 ? 8 : 0) : 0))); + td->words[3] = 0xffffff7f; // retired, all other status/progress bits set + s->buffer[0] += 100; + s->buffer[1] ^= 0x1f; // hardware advances OUT split count/position + } + iso_td_unlink(ep); + ep->armed = false; + } + } + } +} + +static void test_late_schedule_phase(void) { + uint32_t const starts[] = {800, 16383, 0xfffffffe}; + uint32_t const delays[] = {0, 1, 19, 1031}; + for (unsigned interval = 1; interval <= 16; interval++) { + for (unsigned s = 0; s < TU_ARRAY_SIZE(starts); s++) { + for (unsigned d = 0; d < TU_ARRAY_SIZE(delays); d++) { + reset(TUSB_SPEED_HIGH); + assert(open_ep(0x81, TUSB_SPEED_HIGH, 64, interval)); + iso_ep_t* ep = iso_ep_find(1, 0x81); + uint32_t const now = starts[s]; + ehci_data.iso_uframe = now; + ehci_data.iso_last_frindex = now & 0x3fff; + regs.frame_index = now & 0x3fff; + ep->uframe = now - delays[d]; + // Reference: advance one endpoint interval at a time, including wrap. + uint32_t expected = ep->uframe; + while ((int32_t)(expected - (now + 1)) < 0) { + expected += ep->interval; + } + assert(iso_xfer(0, ep, buffer, 64)); + assert(ep->uframe == expected); + } + } + } +} + +static void test_pool_hs_interval(uint8_t interval) { + // Walk the actual DMA chains while four endpoints reuse TDs across both + // frame-list and extended-clock wrap. This also detects orphaned active TDs. + reset(TUSB_SPEED_HIGH); + for (unsigned i = 0; i < TEST_ISO_STREAM_EP_COUNT; i++) { + assert(open_ep((uint8_t)(0x81 + i), TUSB_SPEED_HIGH, 64, interval)); + } + uint32_t const start = 0xfffffff0; + ehci_data.iso_uframe = start; + ehci_data.iso_last_frindex = start & 0x3fff; + regs.frame_index = start & 0x3fff; + for (unsigned i = 0; i < TEST_ISO_STREAM_EP_COUNT; i++) { + ehci_data.qtd_pool[i].iso.uframe = start; + } + unsigned transferred = 0; + unsigned const ticks = tu_max32(1024, (1u << (interval - 1)) * 8); + for (unsigned tick = 0; tick < ticks; tick++) { + uint32_t const now = start + tick; + regs.frame_index = now & 0x3fff; + ehci_link_t link = ehci_data.period_framelist[(now >> 3) % FRAMELIST_SIZE]; + unsigned visited = 0, completed = 0; + while (!link.terminate && link.type != EHCI_QTYPE_QHD) { + assert(link.type == EHCI_QTYPE_ITD); + assert(++visited <= TEST_ISO_STREAM_EP_COUNT); + iso_td_t* td = (iso_td_t*)(uintptr_t)tu_align32(link.address); + if (td->itd.xact[now & 7].active) { + unsigned owners = 0; + for (unsigned i = 0; i < TEST_ISO_STREAM_EP_COUNT; i++) { + iso_ep_t* ep = &ehci_data.qtd_pool[i].iso; + if (ep->busy && ep->armed && ep->td == td) { + assert(ep->uframe == now); + owners++; + } + } + assert(owners == 1); + td->itd.xact[now & 7].active = 0; + td->itd.xact[now & 7].length = 24; + completed++; + } + link = td->itd.next; + } + assert(link.terminate || link.type == EHCI_QTYPE_QHD); + events = 0; + iso_process(true); + assert(events == completed); + for (unsigned i = 0; i < events; i++) { + assert(terminal_events[i].xfer_complete.result == XFER_RESULT_SUCCESS); + assert(terminal_events[i].xfer_complete.len == 24); + } + transferred += completed; + for (unsigned i = 0; i < TEST_ISO_STREAM_EP_COUNT; i++) { + iso_ep_t* ep = &ehci_data.qtd_pool[i].iso; + if (!ep->busy) { + assert(iso_xfer(0, ep, buffer + i * 128, 24)); + } + } + } + // Immediate completion/refill sustains every microframe even with one TD. + if (interval == 1) { + assert(transferred == (ticks - 1) * TEST_ISO_STREAM_EP_COUNT); + } + assert(transferred != 0); +} + +static void test_pool_fs_interval(uint8_t interval) { + for (unsigned hub = 0; hub < (CFG_TUH_HUB ? 2u : 1u); hub++) { + reset(hub ? TUSB_SPEED_HIGH : TUSB_SPEED_FULL); + for (unsigned i = 0; i < TEST_ISO_STREAM_EP_COUNT; i++) { + if (hub) { + buses[1].hub_addr = 2; + buses[1].hub_port = 3; + buses[2].speed = TUSB_SPEED_HIGH; + } + assert(open_ep((uint8_t)(0x81 + i), TUSB_SPEED_FULL, 64, interval)); + } + uint32_t const start = iso_now(); + unsigned transferred = 0; + unsigned const ticks = tu_max32(512, (1u << (interval - 1)) * 64); + for (unsigned tick = 0; tick < ticks; tick += hub ? 1 : 8) { + uint32_t const now = start + tick; + regs.frame_index = (now + (hub ? 0 : 8)) & 0x3fff; + unsigned completed = 0; + if (!hub || (now & 7) == 7) { + ehci_link_t link = ehci_data.period_framelist[(now >> 3) % FRAMELIST_SIZE]; + unsigned visited = 0; + while (!link.terminate && link.type != EHCI_QTYPE_QHD) { + assert(link.type == EHCI_QTYPE_SITD); + assert(++visited <= TEST_ISO_STREAM_EP_COUNT); + iso_td_t* td = (iso_td_t*)(uintptr_t)tu_align32(link.address); + if (td->sitd.active) { + unsigned owners = 0; + for (unsigned i = 0; i < TEST_ISO_STREAM_EP_COUNT; i++) { + iso_ep_t* ep = &ehci_data.qtd_pool[i].iso; + if (ep->busy && ep->armed && ep->td == td) { + assert(ep->uframe == (now & ~7u)); + owners++; + } + } + assert(owners == 1); + td->sitd.active = 0; + td->sitd.total_bytes = 0; + completed++; + } + link = td->itd.next; + } + } + events = 0; + iso_process(true); + if (events != completed) { + fprintf(stderr, "FS pool hub=%u tick=%u now=%u completed=%u events=%u result=%u\n", + hub, tick, now, completed, events, event.xfer_complete.result); + } + assert(events == completed); + for (unsigned i = 0; i < events; i++) { + assert(terminal_events[i].xfer_complete.result == XFER_RESULT_SUCCESS); + } + transferred += completed; + for (unsigned i = 0; i < TEST_ISO_STREAM_EP_COUNT; i++) { + iso_ep_t* ep = &ehci_data.qtd_pool[i].iso; + if (!ep->busy) { + assert(iso_xfer(0, ep, buffer + i * 128, 64)); + } + } + } + assert(transferred != 0); + } +} + +static void test_pool_stream(void) { + for (uint8_t interval = 1; interval <= 16; interval++) { + test_pool_hs_interval(interval); + } +} + +static void test_pool_fs_stream(void) { + for (uint8_t interval = 1; interval <= 16; interval++) { + test_pool_fs_interval(interval); + } +} + +static void test_completion_unlink(bool hs) { + reset(TUSB_SPEED_HIGH); + if (!hs) { + buses[1].hub_addr = 2; + buses[1].hub_port = 1; + buses[2].speed = TUSB_SPEED_HIGH; + } + uint32_t const due = hs ? 801 : 808; + size_t const frame = (due >> 3) % FRAMELIST_SIZE; + unsigned const status_word = hs ? 1 + (due & 7) : 3; + uint32_t const active = TU_BIT(hs ? 31 : 7); + unsigned const type = hs ? EHCI_QTYPE_ITD : EHCI_QTYPE_SITD; + ehci_link_t const original = ehci_data.period_framelist[frame]; + assert(open_ep(0x81, hs ? TUSB_SPEED_HIGH : TUSB_SPEED_FULL, 64, 1)); + assert(open_ep(0x82, hs ? TUSB_SPEED_HIGH : TUSB_SPEED_FULL, 64, 1)); + iso_ep_t* first = iso_ep_find(1, 0x81); + iso_ep_t* second = iso_ep_find(1, 0x82); + assert(iso_xfer(0, first, buffer, 24)); + assert(iso_xfer(0, second, buffer + 64, 24)); + iso_td_t* first_td = first->td; + iso_td_t* second_td = second->td; + first_td->words[status_word] &= ~active; + regs.frame_index = due; + iso_process(true); + assert(events == 1 && !first->busy && !first->armed); + // Completion removes a middle TD immediately, preserving the active sibling. + assert(second_td->words[status_word] & active); + assert(second_td->itd.next.address == original.address); + assert(ehci_data.period_framelist[frame].address == ((uint32_t)(uintptr_t)second_td | (type << 1))); + + uint32_t const next_due = hs ? 802 : 816; + size_t const next_frame = (next_due >> 3) % FRAMELIST_SIZE; + ehci_link_t const next_original = hs ? original : ehci_data.period_framelist[next_frame]; + assert(iso_xfer(0, first, buffer, 24)); + iso_td_t* replacement = first->td; + assert(first->armed && first->uframe == next_due); + second_td->words[status_word] &= ~active; + iso_process(true); + assert(events == 2 && !second->busy); + // Preserve the replacement, including when it precedes the completed HS TD. + unsigned const next_status_word = hs ? 1 + (next_due & 7) : 3; + assert(replacement->words[next_status_word] & active); + assert(replacement->itd.next.address == next_original.address); + replacement->words[next_status_word] &= ~active; + regs.frame_index = next_due; + iso_process(true); + assert(events == 3 && !first->busy); + assert(ehci_data.period_framelist[frame].address == original.address); + assert(ehci_data.period_framelist[next_frame].address == next_original.address); +} + +static void test_future_completion(void) { + uint32_t const starts[] = {800, 0xfffffff8}; + for (unsigned mode = 0; mode < (CFG_TUH_HUB ? 3u : 2u); mode++) { + for (unsigned i = 0; i < TU_ARRAY_SIZE(starts); i++) { + reset(mode == 1 ? TUSB_SPEED_FULL : TUSB_SPEED_HIGH); + if (mode == 2) { + buses[1].hub_addr = 2; + buses[1].hub_port = 1; + buses[2].speed = TUSB_SPEED_HIGH; + } + assert(open_ep(0x81, mode == 0 ? TUSB_SPEED_HIGH : TUSB_SPEED_FULL, 64, 1)); + iso_ep_t* ep = iso_ep_find(1, 0x81); + ep->uframe = starts[i]; + ehci_data.iso_uframe = starts[i] + ehci_data.iso_frame_offset; + regs.frame_index = ehci_data.iso_last_frindex = ehci_data.iso_uframe & 0x3fff; + assert(iso_xfer(0, ep, buffer, 64)); + assert(ep->armed); + uint32_t const due = ep->uframe; + regs.frame_index = (due - 1 + ehci_data.iso_frame_offset) & 0x3fff; + iso_process(true); + assert(events == 0 && ep->busy); + + // Once the interval starts, inspect status and allow immediate completion. + regs.frame_index = (due + ehci_data.iso_frame_offset) & 0x3fff; + iso_process(true); + assert(events == 0); + iso_td_t* td = ep->td; + if (mode == 0) { + td->itd.xact[due & 7].active = 0; + } else { + td->sitd.active = 0; + td->sitd.total_bytes = 0; + } + iso_process(true); + assert(events == 1 && !ep->busy); + assert(event.xfer_complete.result == XFER_RESULT_SUCCESS && event.xfer_complete.len == 64); + } + } +} + +#if CFG_TUH_HUB +// The split allocator rejects sizes outside its documented range. +static void test_split_slot_range(void) { + uint8_t slot = 0xa5; + assert(!hub_iso_split_slot(0, 0, TUSB_DIR_IN, &slot)); + assert(!hub_iso_split_slot(0, 565, TUSB_DIR_IN, &slot)); + assert(!hub_iso_split_slot(0, 800, TUSB_DIR_IN, &slot)); + assert(!hub_iso_split_slot(0, 1024, TUSB_DIR_OUT, &slot)); + assert(!hub_iso_split_slot(0, 1200, TUSB_DIR_OUT, &slot)); + assert(slot == 0xa5); + assert(hub_iso_split_slot(0, 564, TUSB_DIR_IN, &slot) && slot == 1); + assert(hub_iso_split_slot(0, 1023, TUSB_DIR_OUT, &slot) && slot == 0); +} +#endif + +static bool td_linked(iso_ep_t const* ep) { + ehci_link_t const* link = &ehci_data.period_framelist[(ep->uframe >> 3) % FRAMELIST_SIZE]; + while (!link->terminate) { + if (tu_align32(link->address) == (uint32_t) (uintptr_t) ep->td) { + return true; + } + link = list_next(link); + } + return false; +} + +// A slot that passed unvisited is retired without stopping the periodic +// schedule, but not while the controller may be revisiting its entry. +static void test_missed_slot_without_stop(void) { + reset(TUSB_SPEED_HIGH); + init_periodic_list(0); + assert(open_ep(0x81, TUSB_SPEED_HIGH, 64, 1)); + iso_ep_t* ep = iso_ep_find(1, 0x81); + set_now(1000); + assert(iso_xfer(0, ep, buffer, 64) && ep->armed && ep->uframe == 1001 && td_linked(ep)); + uint32_t const slot = ep->uframe; + + // A ring later the controller is back on this microframe: retry later. + set_now(slot + FRAMELIST_SIZE * 8); + iso_process(true); + assert(events == 0 && ep->busy && ep->armed); + set_now(slot + FRAMELIST_SIZE * 8 + 1); + iso_process(true); + assert(events == 1 && !ep->busy && !ep->armed && event.xfer_complete.result == XFER_RESULT_FAILED); + assert(!td_linked(ep) && !ep->td->itd.xact[slot & 7].active); + assert(!regs.command_bm.periodic_enable); // the schedule was never toggled +} + +// Native FS: the embedded TT works on an siTD one frame after FRINDEX reaches +// its entry, so a missed siTD is retired only from the third frame on. +static void test_missed_fs_slot_margin(void) { + reset(TUSB_SPEED_FULL); + init_periodic_list(0); + assert(open_ep(0x81, TUSB_SPEED_FULL, 192, 1)); + iso_ep_t* ep = iso_ep_find(1, 0x81); + set_now(1000); + assert(iso_xfer(0, ep, buffer, 192) && ep->armed); + uint32_t const frame = ep->uframe; + set_now(frame + 8); // slot over, but the controller may still be on the TD + iso_process(true); + assert(events == 0 && ep->armed); + set_now(frame + 16); + iso_process(true); + assert(events == 1 && !ep->armed && event.xfer_complete.result == XFER_RESULT_FAILED); + assert(!td_linked(ep) && !ep->td->sitd.active); +} + +// Abort cancels only a request the controller cannot be about to execute. +static void test_abort_idle_margin(void) { + reset(TUSB_SPEED_HIGH); + init_periodic_list(0); + assert(open_ep(0x81, TUSB_SPEED_HIGH, 64, 1)); + iso_ep_t* ep = iso_ep_find(1, 0x81); + set_now(1000); + assert(iso_xfer(0, ep, buffer, 64) && ep->armed && ep->uframe == 1001); + assert(!iso_stop(0, ep, false)); // next microframe: leave it to complete + assert(ep->busy && ep->armed && td_linked(ep)); + ep->td->itd.xact[ep->uframe & 7].active = 0; + set_now(1001); + iso_process(true); + assert(events == 1 && !ep->busy); + + ep->uframe = 1004; // two microframes ahead once scheduled from 1002 + set_now(1002); + assert(iso_xfer(0, ep, buffer, 64) && ep->armed && ep->uframe == 1004); + assert(iso_stop(0, ep, false) && !ep->busy && !ep->armed && !td_linked(ep)); + assert(events == 1 && !regs.command_bm.periodic_enable); + + // Close while idle frees without touching the schedule. + ep->uframe = 1010; + assert(iso_xfer(0, ep, buffer, 64) && ep->armed); + uint32_t const entry = (ep->uframe >> 3) % FRAMELIST_SIZE; + assert(iso_stop(0, ep, true) && iso_ep_find(1, 0x81) == NULL); + assert(ehci_data.period_framelist[entry].terminate || ehci_data.period_framelist[entry].type == EHCI_QTYPE_QHD); + assert(!regs.command_bm.periodic_enable); +} + +// Schedule helpers are bounded and accept a halted controller. +static void test_schedule_helpers_bounded(void) { + reset(TUSB_SPEED_HIGH); + regs.command_bm.periodic_enable = 1; + regs.status_bm.periodic_status = 1; // never follows: the wait must give up + assert(!ehci_disable_schedule(®s, true)); + regs.status_bm.hc_halted = 1; // a halted controller runs nothing + assert(ehci_disable_schedule(®s, true)); + regs.status_bm.hc_halted = 0; + regs.status_bm.periodic_status = 0; + regs.command_bm.periodic_enable = 0; + assert(ehci_disable_schedule(®s, true)); + ehci_enable_schedule(®s, true); + assert(regs.command_bm.periodic_enable); +} + +// Cancelling a queued request keeps the endpoint's service phase. +static void test_abort_keeps_phase(void) { + reset(TUSB_SPEED_HIGH); + assert(open_ep(0x81, TUSB_SPEED_HIGH, 64, 3)); // 4-microframe interval + iso_ep_t* ep = iso_ep_find(1, 0x81); + set_now(1000); + ep->uframe = 1202; // phase 2, beyond the ring so the request stays unarmed + assert(iso_xfer(0, ep, buffer, 64) && !ep->armed); + assert(iso_stop(0, ep, false) && !ep->busy); + assert(iso_xfer(0, ep, buffer, 64)); + assert(ep->uframe % ep->interval == 2); + assert(iso_stop(0, ep, false)); +} + +// Reopening an open ISO endpoint succeeds only with unchanged parameters. +static void test_reopen_parameters(void) { + reset(TUSB_SPEED_HIGH); + assert(open_ep(0x81, TUSB_SPEED_HIGH, 64, 1)); + iso_ep_t* ep = iso_ep_find(1, 0x81); + assert(open_ep(0x81, TUSB_SPEED_HIGH, 64, 1) && iso_ep_find(1, 0x81) == ep); + assert(!open_ep(0x81, TUSB_SPEED_HIGH, 192, 1)); // larger packet + assert(!open_ep(0x81, TUSB_SPEED_HIGH, 64, 4)); // other interval + assert(ep->max_xfer_bytes == 64 && ep->interval == 1); + iso_ep_free(ep); // close without the DMA stop handshake the fixture cannot emulate + assert(iso_ep_find(1, 0x81) == NULL); + assert(open_ep(0x81, TUSB_SPEED_HIGH, 192, 4)); +} + +// The scheduler never enables SOF: completions and frame-list rollover drive it. +static void test_sof_free(void) { + reset(TUSB_SPEED_HIGH); + regs.inten |= EHCI_INT_MASK_FRAMELIST_ROLLOVER; // as ehci_init() enables it + assert(open_ep(0x81, TUSB_SPEED_HIGH, 64, 1)); + assert(open_ep(0x02, TUSB_SPEED_HIGH, 64, 8)); // 128-microframe interval + iso_ep_t* in = iso_ep_find(1, 0x81); + iso_ep_t* out = iso_ep_find(1, 0x02); + + // A slot whose phase already passed moves forward, keeping its phase. + set_now(1000); + in->uframe = 990; + assert(iso_xfer(0, in, buffer, 64)); + assert(in->armed && in->uframe == 1001); + + // A TD beyond the current ring waits for rollover to arm it. + out->uframe = 1000 + 2 * FRAMELIST_SIZE * 8 + 5; + assert(iso_xfer(0, out, buffer + 64, 64) && !out->armed); + // Its frame is still beyond the ring at the next rollover... + set_now((1000 & ~(FRAMELIST_SIZE * 8 - 1)) + FRAMELIST_SIZE * 8); + in->td->itd.xact[in->uframe & 7].active = 0; + regs.status = EHCI_INT_MASK_FRAMELIST_ROLLOVER; + hcd_int_handler(0, true); + assert(events == 1 && !in->busy); // rollover also retired the visited TD + assert(event.xfer_complete.result == XFER_RESULT_SUCCESS); + assert(!out->armed); + // ...and inside it at the following one, armed in its original slot. + uint32_t const due = out->uframe; + set_now(iso_now() + FRAMELIST_SIZE * 8); + regs.status = EHCI_INT_MASK_FRAMELIST_ROLLOVER; + hcd_int_handler(0, true); + assert(out->armed && out->uframe == due); + assert(ehci_data.period_framelist[(due >> 3) % FRAMELIST_SIZE].address == + ((uint32_t) (uintptr_t) out->td | (EHCI_QTYPE_ITD << 1))); + + // A completion retires it; SOF was never enabled. + set_now(due); + out->td->itd.xact[due & 7].active = 0; + iso_process(true); // as the completion interrupt does + assert(events == 2 && !out->busy && event.xfer_complete.result == XFER_RESULT_SUCCESS); + assert(!(regs.inten & EHCI_INT_MASK_NXP_SOF)); +} + +#endif + +int main(void) { + // Hardware links are 32-bit. The runner places static fixtures below 4 GiB. + assert((uintptr_t)&ehci_data <= UINT32_MAX && (uintptr_t)buffer <= UINT32_MAX); + test_attach_debounce(); + test_endpoint_lookup(); + test_qtd_retirement(); +#if EHCI_ISO_ENABLED + test_shared_pools(); + test_sparse_iso_pool(); + test_native_fs(); +#if CFG_TUH_HUB + test_split(); + test_split_audio(); + test_split_allocation(); +#endif + test_hs(); + test_iso_status_errors(); + test_iso_clock_config(); + test_long_interval_and_wrap(); + test_limits_and_late_completion(); + test_schedule_sweep(); + test_descriptor_reuse(); + test_late_schedule_phase(); + test_pool_stream(); + test_pool_fs_stream(); + test_completion_unlink(true); +#if CFG_TUH_HUB + test_completion_unlink(false); +#endif + test_future_completion(); + test_sof_free(); + test_missed_slot_without_stop(); + test_missed_fs_slot_margin(); + test_abort_idle_margin(); + test_schedule_helpers_bounded(); + test_abort_keeps_phase(); + test_reopen_parameters(); +#if CFG_TUH_HUB + test_split_slot_range(); +#endif +#else + reset(TUSB_SPEED_HIGH); + tusb_desc_endpoint_t const iso_desc = { + .bLength = sizeof(iso_desc), .bDescriptorType = TUSB_DESC_ENDPOINT, + .bEndpointAddress = 0x81, .bmAttributes = {.xfer = TUSB_XFER_ISOCHRONOUS}, + .wMaxPacketSize = 64, .bInterval = 1 + }; + assert(!hcd_edpt_open(0, 1, &iso_desc)); + assert(regs.command_bm.int_threshold == 8); +#endif +#if CFG_TUH_MEM_DCACHE_ENABLE + // Cached payloads must still exercise clean, invalidate and clean/invalidate. + for (unsigned i = 0; i < TU_ARRAY_SIZE(test_cache_calls); i++) { + assert(test_cache_calls[i] > 0); + } +#endif + puts("EHCI ISO regression tests passed"); + return 0; +}
diff --git a/test/unit-test/host/ehci/tusb_config.h b/test/unit-test/host/ehci/tusb_config.h new file mode 100644 index 0000000..48cfd25 --- /dev/null +++ b/test/unit-test/host/ehci/tusb_config.h
@@ -0,0 +1,20 @@ +// SPDX-License-Identifier: MIT +#ifndef TUSB_CONFIG_H_ +#define TUSB_CONFIG_H_ +#define CFG_TUSB_MCU OPT_MCU_NONE +#define CFG_TUSB_OS OPT_OS_NONE +#define CFG_TUSB_DEBUG 0 +#define CFG_TUH_ENABLED 1 +#define CFG_TUH_MAX_SPEED OPT_MODE_HIGH_SPEED +#define CFG_TUH_DEVICE_MAX 4 +#define CFG_TUH_ENDPOINT_MAX (CFG_TUH_HUB ? 8 : 9) // Keep multiple pool bitmap words without a hub. +#ifndef CFG_TUH_HUB +#define CFG_TUH_HUB 1 +#endif +#define TUP_USBIP_EHCI +// Simulated DMA memory is coherent; descriptors need no cache maintenance. +#define CFG_TUH_UNCACHED_MEM_SECTION +#ifndef TEST_EHCI_GENERIC +#define TUP_USBIP_CHIPIDEA_HS +#endif +#endif