3.3. ICSS EtherNet/IP Adapter Developer Guide¶

3.3.6.2.1. Forwarding Modes¶

1. Cut-Through : Data is received on Rx FIFO and copied directly to Tx FIFO. There are no queues involved. It’s the fastest mode of data transfer since no data copying is involved.
2. Delayed Cut-Through : Data is first received into the port queues (Refer to EMAC Design guide) and then forwarded. This is a slower operation compared to cut-through but allows full inspection of the packet.

The key differences are listed below.

As always the switch needs to decide when to forward the packets and how, this is done inside the firmware by waiting for a certain number of bytes and then applying the forwarding rules. The number of bytes firmware waits for and the forwarding rules are different for each protocol example. For instance for EtherNet/IP this is 22 bytes because some DLR packet information is available only on the 22nd byte.

The number of bytes firmware waits for decides the resident time for a packet inside the switch. Since the PRU is a fully deterministic architecture this is a known number. The illustration below explains how this is calculated.

Using the calculation above, resident delay for a cut-through frame on EIP firmware is 22 * 80 + 640 + 160 = 2560ns or 2.5 us. The resident delay for a delayed cut-through packet varies depending on scheduling delays. For a 66 byte PTP frame the delay is around 9-10 us based on experimental results.

3.3.6.2.2. General Forwarding Rules¶

1. Broadcast frames : Frames are first run through the Storm Prevention check and then get cut-through (subject to link availability on other port) as well as get forwarded to Host.
2. Multicast frames : DLR and PTP frames bypass Storm Prevention check and have their own rules of forwarding which will be discussed later. Other frames are treated like Broadcast (forward to host and cut-through)
3. Unicast frame to Host : UC frames matching interface MAC get forwarded to Host
4. Unicast frames not to Host : UC frames not matching interface MAC get cut-through
5. Cut-Through decision but other port is occupied : Frame gets delayed cut-through

3.3.6.3.1. Frame Types¶

1. Beacon : Is the most common frame type and used to monitor ring health. It is sent out periodically on both the ports by the Supervisor based on the ring parameter beaconInterval. The lowest possible value for this is 100us. The beacons are transmitted simultaneously from both ports and reach the opposite ports of same device after traversing the entire network, this is how the supervisor confirms that ring is functional. Failure to observe a beacon within a specified interval called beaconTimeout triggers a state change to Fault. This type of frame is cut-through.
2. Announce : Analogous to Beacon but much slower, this is for legacy networks. EIP Adapter does not support this mode. All Announce frames get cut-through.
3. Sign On : This contains information about every node on the network. This is sent on one port by the Supervisor and gets trapped by every node on the network, the devices then add their own information and send it over the other port. These frames are sent to the Host over Host queues, the Host adds it’s own information and then transmits it over the other port.
4. Locate Fault : This is a frame sent by the Supervisor in response to a missed beacon or any other fault event. All devices are supposed to report their neighbor status on receiving this frame. EIP Adapter cuts-through this frame and generates Neighbor Check Requests for neighboring devices.
5. Neighbor Check Request : This is a frame that is generated by every node including the supervisor in response to a fault generating event on the network. A device is supposed to consume this frame and generate a neighbor check request. Our application forwards this frame to Host for a response. A device generates 3 such frames on each port at intervals of 100ms and if no response is received then a status is reported to Supervisor that corresponding port has an issue.
6. Neighbor Check Response : A device is supposed to consume this frame. If this frame isn’t received after 3 requests then a neighbor status frame is sent indicating the port on which device is faulty. This frame is not forwarded.
7. Link Status/Neighbor Status : This is sent by a device to supervisor to indicate a link break or an issue with adjacent device. This frame is cut-through.

3.3.6.3.2. Timers¶

DLR uses lots of timers because it has to keep track of when every beacon arrives and whether there has been a timeout, there’s also a timeout related to neighbor check response. The lowest beacon interval according to the standard (and what the application supports) is 100us and with a minimum 2x timeout interval the minimum value for timeout comes to 200us. Thus the minimum resolution for a DLR Beacon timeout timer should be 200us. This is only possible with either watchdog timers (internal to PRUSS) or DMTimers (on the SoC).

The DMTimer ISR is configured to get triggered on expiry of the count (counts downwards). Every time a beacon is received in firmware the timer is triggered to restart, this prevents the ISR from getting hit.

For the neighbor check response timers we use software timers since the requirement is very lax (100 ms). SYS/BIOS provides API’s for configuring software timers which are used here, if porting to another operating system this must be taken care of.

The timers and their ISR’s are listed below

1. beaconTimerPort0()
   1. Beacon timeout timer for Port 0.
   2. Timer Handle is dlrTimer_PORT0
2. beaconTimerPort1()
   1. Beacon timeout timer for Port 1.
   2. Timer Handle is dlrTimer_PORT1
3. neighborTimeoutISR0()
   1. ISR for Neighbor Timeout Timer on Port0.
   2. Timer Handle name is dlrNeighborTimeoutClock[0]
4. neighborTimeoutISR1()
   1. ISR for Neighbor Timeout Timer on Port1.
   2. Timer Handle name is dlrNeighborTimeoutClock[1]

The timers with their default values are configured in the DLR initialization API initDLR(). The beacon timeout timers are configured with network parameters received from the master when the first beacon is received.

3.3.6.3.3. Interrupts¶

There are four interrupt contexts to DLR

1. Related to timers : When timers expire they trigger an ISR. Total of four timer related ISR’s are applicable to DLR. They are covered above.
2. Related to link break : Certain DLR events need to be processed when a link break occurs. EMAC LLD has one link ISR per port to handle this. A DLR callback is registered for the link ISR which is specific to this. There is one callback per port, these are identical and only differ in terms of which port they belong to.
   1. EIP_DLR_Port0ProcessLinkBrk() Link break callback for Port 0.
   2. EIP_DLR_Port1ProcessLinkBrk() Link break callback for Port 1.
3. Related to state change processing DLR mostly uses a fast ISR scheme for communicating with the host where firmware parses the fields, copies the data to preset locations and triggers an interrupt for the ARM. There is one ISR for each port. The acceptable latency of the ISR is determined by the beacon interval. If running at minimum level i.e. 100us, the ISR must be triggered and completed within that time.
   1. EIP_DLR_Port0ISR() : Fast ISR for Port 0
   2. EIP_DLR_Port1ISR() : Fast ISR for Port 1
4. Related to standard Rx interrupt which uses the Host queues : Some DLR packets need to be processed by the Host and as per the specification are sent to the highest priority queue. They are handled by a DLR specific callback. This callback is the function EIP_DLR_processDLRFrame()

3.3.6.3.4. State Machine and DLR Actions¶

A DLR Ring has three states (two as per standard). The state information is stored in shared RAM at the location pointed by DLR_STATE_MACHINE_OFFSET (See memory map for details)

1. No DLR : This is the state right after boot up and in the absence of any DLR frames. It’s not an official classification, at this stage the beacon timeout timers are configured with default values and all flags are in a reset state. Corresponds to DLR_IDLE_STATE_VAL
2. Fault : When the first beacon is received this is the state of state machine. A field in the DLR frame corresponds to this and is set by the Supervisor. When a supervisor receives two beacon frames that have traversed the entire network and reached opposing ports it changes this state to Normal. A fault state also occurs when a collision is detected or network is reset. When the first DLR beacon is received and the state machine is in reset state the DLR beacon timeout interval is copied and the DLR timeout timer is configured with the value in the Fast ISR. Corresponds to the value DLR_FAULT_STATE_VAL
3. Normal : Indicates that ring is up and functional. This corresponds to the value DLR_NORMAL_STATE_VAL

Besides the state machine two other variables are of importance.

1. Events common to both ports : DLR_COMMON_EVENTS_OFFSET is the memory location which corresponds to this. The values stored in this variable and the actions corresponding to it are
   1. DLR_RING_NORMAL_TRANSITION  : Node has transitioned from Fault to Normal. Perform appropriate actions. Look for the flag DLR_RING_NORMAL_TRANSITION_MASK in the code.
   2. DLR_RING_FAULT_TRANSITION  : Node has transitioned from Reset to Fault. Configure the timers. Look for the flag DLR_RING_FAULT_TRANSITION_MASK
   3. DLR_STOP_BOTH_TIMERS  : Stop both beacon timeout timers.
   4. DLR_PORT0_BEACON_RCVD : First beacon received for Port 0
   5. DLR_PORT1_BEACON_RCVD : First beacon received for Port 1
   6. DLR_RESET_EVENT_OCCURED : Reset the state machine
2. Events symmetrical but unique to each port : The memory location corresponding to this is DLR_PORT_EVENTS_OFFSET. Common events are
   1. DLR_LOCFAULT_RCVD  : Locate Fault Packet received.
   2. DLR_NCREQ_RCVD  : Neighbor Check request packet received
   3. DLR_NCRES_RCVD  : Neighbor Check response packet received
   4. DLR_START_TIMER  : Start the beacon timeout timer corresponding to the port
   5. DLR_RING_FAULT_RCVD  : Beacon received with ring status set to fault, transition to fault state
   6. DLR_DROP_PACKET  : Internal to firmware implementation
   7. DLR_UPDATE_SUP_CFG  : Supervisor has changed, update params
   8. DLR_HOST_FWD  : Internal to firmware implementation
   9. DLR_TIMER_RUNNING  : When beacon timeout timer is started this is set
   10. IS_A_DLR_FRAME  : Internal to firmware implementation
   11. COMPARE_SUP_PRED  : Internal to firmware implementation
   12. UPDATE_NEW_SUP_CFG  : Internal to firmware implementation
   13. DLR_FLUSH_TABLE_RCVD  : Flush table packet received. Flush learning table
   14. DLR_SEND_LEARNING_UPDATE  : Send learning update frame.

3.3.6.3.5. QoS¶

All DLR frames when they are sent to Host go on the highest priority queue. A callback function EIP_DLR_processDLRFrame() in the EIP application for the highest priority queue specifically handles this.

3.3.6.3.6. Interface¶

All DLR information relevant to the stack is stored in the structure dlrStruct which maps directly to the DLR object as defined in the standard, attribute ID’s refer to the corresponding ID’s for DLR Object. The macro IS_A_DLR_SUPERVISOR has been kept for future use and cannot be enabled right now.

The structure is used as a global in icss_dlr.c. A third party stack can declare this as an extern to access the member variables.

/**
* @brief DLR parent structure through which all other structures can be accessed
*/
typedef struct
{
    /**Supervisor config. Attribute ID 4*/
    superConfig supConfig;
    /**Supervisor address. Attribute ID 10*/
    activeSuperAddr addr;
    /**State Machine variables. Attributes 1 through 3*/
    dlrStateMachineVar SMVariables;
    #ifdef IS_A_DLR_SUPERVISOR
    /**Attribute ID's 6 and 7*/
    lastActiveNode activeNode[2];
    #endif
    /**DLR Capabilities, flag
    The Map is as follows
    * 0 : Announce Based Ring Node
    * 1 : Beacon based Ring Node
    * 2-4 : Reserved
    * 5 : Supervisor capable
    * 6 : Redundant capable
    * 7 : Flush Table frame capable
    * 8-31 : Reserved
    */
    /**DLR Capabilities of the device. Attribute ID 12*/
    uint32_t dlrCapabilities;
    /**Active Supervisor precedence. Attribute ID 11*/
    uint8_t activeSuperPred;
    /**DLR Port 0 Interrupt number for ARM*/
    uint8_t port0IntNum;
    /**DLR Port 1 Interrupt number for ARM*/
    uint8_t port1IntNum;
    #ifdef IS_A_DLR_SUPERVISOR
    /**Number of Ring Faults since power up. Attribute ID 5*/
    uint32_t numRingFaultsPowerUp;
    /**Number of ring participants. Attribute ID 8*/
    uint16_t ringParticipantsCount;
    /**pointer to array of structures. Attribute ID 9*/
    protocolParticipants **ringNodes;
    #endif
} dlrStruct;

3.3.6.3.7. Initialization¶

DLR is initialized using the API initDLR() and started with the API call startDLR(). A call to the API stopDLR() disables DLR feature in the firmware. Initialization needs to be done only on startup. DLR can be enabled and disabled at runtime.

The IP address for DLR node is configured through the API addDLRModuleIPAddress(uint32_t ipAddress). Developer needs to call this once an IP address has been acquired or else DLR frames will not carry an IP address.

3.3.6.4.1. Host Receive queues¶

• Queue 1 :
  • PTP and DLR frames are received on this queue.
  • This is called the Real Time (RT) queue and is linked to the RT callback rxRTCallBack
  • If a frame isn’t DLR or PTP it goes to the TCP/IP stack
• Queue 2 : Same behavior as Queue 1
• Queue 3 : Same behavior as Queue 1 and 2
• Queue 4 : Mapped to NDK

3.3.6.4.2. Transmit queues¶

• Queue 1
  • PTP and DLR frames are sent on this frame
• Queue 2 : Regular queue. No mapping
• Queue 3 : Same as Queue 2
• Queue 4 : Same as Queue 3

3.3.11.1.1. Board and Hardware Initialization¶

This part is application specific. In Processor SDK the main API to call is Board_Init() with predefined flags which initialize each specific HW module of choice. For example in EtherNet/IP adapter application the modules which are configured by this are

• Pin Mux
• Timers
• UART

The following code performs this task

Board_init(BOARD_INIT_PINMUX_CONFIG | BOARD_INIT_MODULE_CLOCK | BOARD_INIT_UART_STDIO);

Some modules like I2C’s are not initialized by this. They are initialized with the call

Board_i2cLedInit();

For more details about pinmux and hardware specific initialization please consult the PRU-ICSS/PRU_ICSSG Migration Guide.

3.3.11.1.2. Mapping the Interrupts¶

The EtherNet/IP adapter uses the following PRU-ICSS Interrupts

The interrupt numbers are configured at the start of application, this along with the interrupt mapping contained in tiswitch_pruss_intc_mapping.h is used to program the PRUSS Interrupt controller and ARM interrupt controller. The interrupt mapping is explained here

Sample interrupt initialization code from EtherNet/IP example provided below

/* Interrupt configuration */
switchEmacCfg->rxIntNum = icssInterruptOffset + EIP_RX_INT_OFFSET;
switchEmacCfg->linkIntNum = icssInterruptOffset + EIP_LINK_INT_OFFSET;

/*Configure Time Sync interrupts*/
timeSyncHandle->timeSyncConfig.txIntNum = icssInterruptOffset + EIP_PTP_INT_OFFSET;

/*Configure DLR*/
dlrHandle->dlrObj->port0IntNum = icssInterruptOffset + EIP_DLR_P0_INT_OFFSET;
dlrHandle->dlrObj->port1IntNum = icssInterruptOffset + EIP_DLR_P1_INT_OFFSET;
dlrHandle->dlrObj->beaconTimeoutIntNum_P0 = icssInterruptOffset + EIP_BEACON_TIMEOUT_INT_OFFSET_P0;
dlrHandle->dlrObj->beaconTimeoutIntNum_P1 = icssInterruptOffset + EIP_BEACON_TIMEOUT_INT_OFFSET_P1;

3.3.11.1.3. Allocating Memory for Structues¶

Application explicitly allocates memory only for the ICSS EMAC Handle and the Switch configuration. This is done by the following code

/*Create Handle*/
prusshandle = PRUICSS_create(pruss_config, PRUICSS_INSTANCE);

/*Allocate memory for Handle*/
emachandle = (ICSS_EmacHandle)malloc(sizeof(ICSS_EmacConfig));

/*Switch configuration*/
ICSSEMAC_InitConfig *switchEmacCfg;

/*Allocate memory for config structure*/
switchEmacCfg = (ICSSEMAC_InitConfig *)malloc(sizeof(ICSSEMAC_InitConfig));

Memory for specific modules is allocated and handled by the respective module initialization API’s. For example EIP_DLR_init() and TimeSync_drvInit() handle initialization for DLR and PTP respectively.

3.3.11.1.4. Registering the callbacks¶

The standard callbacks which are common to all switch implementations are allocated by the API EIPAPP_initICSSEmac() with the following

::

ICSS_EmacBaseAddressHandle_T emacBaseAddr; ICSS_EmacCallBackObject *callBackObj;

/* Callback mallocs */ callBackObj = (ICSS_EmacCallBackObject *)malloc(sizeof(

ICSS_EmacCallBackObject));

callBackObj->learningExCallBack = (ICSS_EmacCallBackConfig *)malloc(sizeof(

ICSS_EmacCallBackConfig));

callBackObj->rxCallBack = (ICSS_EmacCallBackConfig *)malloc(sizeof(

ICSS_EmacCallBackConfig));

callBackObj->txCallBack = (ICSS_EmacCallBackConfig *)malloc(sizeof(

ICSS_EmacCallBackConfig));

callBackObj->rxRTCallBack = (ICSS_EmacCallBackConfig *)malloc(sizeof(

ICSS_EmacCallBackConfig));

((ICSS_EmacObject *)handle->object)->callBackHandle = callBackObj;

• Learning callback is used by DLR to disable learning of Supervisor MAC ID. This is a protocol specific requirement. It is set to NULL by default and is mapped to the API checkSupervisorException() in the DLR Initialization API.
• Generic Tx and Rx callbacks are mapped to the default TCP/IP Transmit and Receive API’s respectively inside the EMAC-LLD.
• RT callback is meant for the highest priority queue or which ever queue is configured exclusively for protocol specific frames. Please see the `QoS`_ section for how it works with EtherNet/IP.

For protocol specific callbacks a separate callback is registered which overwrites the default and assigns the Real Time callback to a protocol specific API. In EtherNet/IP this API is EIP_processProtocolFrames which is a combined callback for PTP and DLR frames. The callback EIP_processProtocolFrames() copies the frame to a temporary buffer and parses the protocol field to check if it’s a PTP or DLR frame before passing it to the appropriate handler.

Callback assignment for EtherNet/IP in eip_driver_init()

/*Packet processing callback*/
  ((((ICSS_EmacObject *)
    icssEmacHandle->object)->callBackHandle)->rxRTCallBack)->callBack =
       (ICSS_EmacCallBack)EIP_processProtocolFrames;
  ((((ICSS_EmacObject *)
    icssEmacHandle->object)->callBackHandle)->rxRTCallBack)->userArg =
       icssEipHandle;

3.3.11.1.5. Configuring Switch parameters¶

Switch/EMAC parameters are part of the structure ICSSEMAC_InitConfig. The members are explained in EMAC LLD Design guide

For EtherNet/IP configure the parameters as follows.

Common Settings

• portMask`` : set to ``ICSS_EMAC_MODE_SWITCH
• ethPrioQueue`` : set to Queue 3. ``ICSS_EMAC_QUEUE3
• ``halfDuplexEnable`` : Enable half duplex by setting to 1
• ``enableIntrPacing`` : Enable pacing by setting to 1
• intrPacingMode`` : Use pacing mode 1. ``INTR_PACING_MODE1
• ``pacingThreshold`` : Set to 100
• ``learningEn`` : Enable by setting to 1

3.3.12.4.1. Test Setup¶

Conformance Test Tool is installed in Windows Test Machine. The machine is devoid of Firewall and Antivirus software which may interfere with some of the tests. The DUT connected to the Test machine using an Ethernet cable.

Fig. 3.17 Test Setup for Conformance Testing

3.3.12.4.2. Test Procedure¶

1. When the tool is launched, it will show a confirmation message. Click OK to proceed

Fig. 3.18 ODVA CTT Startup Message

1. Select Add Device Option. Browse and select the STC file ({IA_SDK_HOME}\examples\ethernetip_adapter\stc) provided along with the release package. Provide a name for the device and press ‘OK’

Fig. 3.19 Select/Add a new Device

Fig. 3.20 Adding a new Device

1. Make sure that the Adapter related information (Vendor Name, Device Type, Product code etc…) are read and listed properly by the CTT. Give a log file name so that the CTT log is saved in this name.

Fig. 3.21 Configuring Device name and CTT log file name

1. Select Physical data and change the IP Address and MAC address as that of DUT.

Fig. 3.22 Configuring Physical Data to match DUT parameters

1. Now the DUT has been added to the CTT device list. We can edit the parameters as and when required. When more than one device is listed in the device list, select the appropriate one to start the tests.

Fig. 3.23 The newly added device is listed in Device List

1. Click on the |Runtests.png| (or Tools -> Run Tests). In the test window, test window, we can select individual development tests under test mode ‘Development’.

Fig. 3.24 CTT Development Test Window

1. In Conformance Test mode, the tool will automatically select the supported test cases as per the STC file. (Tick ‘Run continuously’ option when continuous iterations are required).

Fig. 3.25 CTT Conformance Test Window

Note

IMPORTANT: Make sure that the NIC of the PC in which the CTT is running is configured properly. Ideally the DUT IP and the IP address of the test machine should belong to the same subnet as shown below.

Fig. 3.26 Test PC should be configured in the same subnet as that of DUT

Note

Additional IP needs to be configured to the test machine before running the CTT tests as follows:

Fig. 3.27 Configuring additional IP

1. When the testing is completed, CTT will show the results which will reveal the number of errors and warnings.

Fig. 3.28 Final Test Results Window (The number of errors may vary)

3.3.12.5.1. Test Setup¶

1. The Molex EIP tool can be downloaded from Molex Website and shall be extracted to Windows PC (Installation not required).
2. Launch EIP_Tools.exe
3. The DUT connected to the Test machine over LAN.

Fig. 3.29 Test setup for EIP tool test

3.3.12.5.2. Test Procedure¶

Fig. 3.30 Selecting the proper NIC

1. Configure the DUT IP address and issue a ‘List Identity Request’ to the DUT. If the DUT is detected, it will be listed properly as TI/Molex EIP Adapter Sample.

Fig. 3.31 DUT is detected properly.

1. Now we can communicate with the DUT using EtherNet/IP Request-Response from different tabs in this tool. Explicite Message – This is used for verifying explicit Request-Response communication between this tool and the DUT. Status will be highlighted with yellow indicating there is/are unsupported services.

Fig. 3.32 Explicit Message Window

Class – We can read the Revision, Maximum instances and number of instances using ‘Get_Attribute_All’. Use Get_Attribute to read individual parameters. Status will be ‘OK’ highlighted with Green when the communication with the DUT is proper.

Fig. 3.33 Class Object Window

0x01 Identity – Use Get_Attribute_All to read all the Identity related information from the EtherNet/IP Adapter. All the supported parameters will be shown in the tool. Get_Attribute will read individual parameters also. Unsupported fields will be read and filled as blank.

Fig. 3.34 0x01 Identity Object Window

0x06 Connection Manager – All the supported Connection Manager related parameters will be read and displayed upon a Get_Attribute_All request from the tool.

Fig. 3.35 0x06 Connection Manager Object Window

0x47 DLR – Device Level Ring (DLR) object attributes are queried and displayed.

Fig. 3.36 0x47 DLR Object Window

0x48 QoS – QoS object attributes values are displayed.

Fig. 3.37 0x48 QOS Object Window

3.3.12.5.3. TCP/IP Object¶

This object is used to read/write DUT interface settings and their configurations. The Configuration Control Attribute (Attr. 3) will show whether the DUT uses permanently stored IP or an IP address assigned via DHCP.

Fig. 3.38 0xF5 TCP/IP Object Window

3.3.12.5.4. Ethernet Link Object¶

Issue a Get_Attibute_All command and all the supported parameters are read and displayed. The Interface Control attribute (Attr. 6) will show the speed and duplicity of the device.

Fig. 3.39 0xF6 Ethernet Link object Window

Fig. 3.40 Forcing the DUT to 100 Mbps mode

Fig. 3.41 Forcing the DUT to 10 Mbps mode