Users of the standard IEC 60601-2-24:2012 (infusion pumps and similar devices) might scratch their heads over some of the details in the latest edition associated with the performance tests. A recent project comparing the first and second edition along with the US version AAMI ID26 might offer a explanation.
The AAMI standard, formally ANSI/AAMI ID26 is a modified version of IEC 60601-2-24:1998. It It includes some dramatic deviations and arguably makes the performance testing significant more complex. This AAMI standard has since been withdrawn and is no longer available for sale, and is not used by the FDA as a recognized standard. But the influence on the 2012 edition of IEC 60601-2-24 is such that it is worth getting a copy of this standard if you can, just to understand what is going on.
It seems that committee working on the second edition of IEC 60601-2024 were seriously looking at incorporating some of the requirements from the AAMI standard, and got as far as deleting some parts, adding some definitions and then … forgot about it and published the standard anyway. The result is a standard that does not make sense in several areas.
For example, in IEC 60601-2-24:1998 the sample interval for general performance tests is fixed at 0.5 min. In the AAMI edition, this is changed to 1/6min (10s). In IEC 60601-2-24:2012, the sample interval is .. blank. It’s as if the authors delete the text “0.5 min”, were about to type in “1/6 min”, paused to think about it, and then just forgot and went ahead and published the standard anyway. Maybe there was a lot of debate about the AAMI methods, which are indeed problematic, followed by huge pressure to get the standard published so that it could be used with the 3rd edition of IEC 60601-1, so they just hit the print button without realising some of the edits were only half baked. Which we all do, but … gee … this is a international standard used in a regulatory context and cost $300 bucks to buy. Surely the publication process contains some kind of plausibility checks?
Whatever the story is, this article looks at key parameters in performance testing, the history, issues and suggests some practical solutions that could be considered in future amendments or editions. For the purpose of this article, the three standards will be called IEC1, IEC2 and AAMI to denote IEC 60601-2-24:1998, IEC 60601-2-24:2012 and ANSI/AAMI ID 26 (R)2013 respectively.
In IEC1, the minimum rate is defined as being the lowest selectable rate but not less than 1mL/hr, with a note that ambulatory use pumps should be the lowest selectable rate. OK, time to be pedantic - according IEC rules, “notes” should be used for explanation only, so strictly speaking the 1mL/hr is the lowest testing rate, end of story.
Obviously we need to consider this note as “normative” as ambulatory pumps usually have rates that are tiny compared to a typical bedside pump - insulin pumps for example have basal (background) rates as low as 0.00025mL/hr, and even the highest speed bolus rates are less than 1mL/hr. So a minimum rate of 1mL/hr makes absolutely no sense.
The AAMI standard tried to deal with this by deleting the note and creating a new defined term - “lowest selectable rate” and then adding a column to Table 102 to include this as one of the rates that can be tested. Slight problem, though, is that they forgot to select this item in Table 102 for ambulatory pumps, the very device that we need it for. Instead, ambulatory pumps must still use the “minimum rate”. But, having deleted the note, it meant that ambulatory pumps have to be tested 1mL/hr - a rate that for insulin (applied continuously) would probably kill an elephant.
While making a mess of the ambulatory side, the tests for other pumps are made unnecessarily complex as they do require tests at the lowest selectable rate in addition to 1mL/hr. Many bedside pumps are adjustable down to 0.1mL which is the resolution of the display. However, no one really expects this rate to be accurate, it’s like expecting your car’s speedometer to be accurate at 1km/hr. It is technically difficult to test, and the graphs it produces are likely to be messy and meaningless to the user. Clearly not well thought out, and appears to be ignored by manufacturers in practice.
IEC2 also tried to tackle the problem. Obviously following AAMI’s lead, they got as far as deleting the note in the minimum rate definition, adding the new defined term (“minimum selectable rate”), … but that’s as far as it got. This new defined term never gets used in the normative part of the standard, The upshot is that ambulatory pumps are again required to kill elephants, with a “minimum rate” of 1mL/hr.
So what is the real world solution? What probably makes the most sense is to avoid fixing it at any number, and leave it up to the manufacturer to decide a “minimum rate” at which pump provides reliable repeatable performance. This rate should of course be declared in the operation manual, and the notes (this time used properly as hints rather than a requirement) can suggest a typical minimum rate of 1mL/hr for general purpose bedside pumps, and highlight the need to assess the risks associated with the user selecting a rate lower than the declared minimum rate.
That might sound a bit wishy washy, but the normal design approach is to have “reliable range of performance” which is greater than what people need in clinical practice. As long as that principle is followed, the fact that the pump can still be set below the minimum rate is not a significant risk. It is only if the manufacturer tries to cheat - in order to make nicer graphs - and select a “minimum rate” that is higher than the clinical needs that the risk starts to become significant. And competition should encourage manufacturers to use a minimum rate that is as low as possible.
In IEC1, there are no performance tests required at the maximum selectable rate. Which on the face of it seems a little weird, since the rate of 25mL/hr is fairly low compared to maximum rates in the 1000mL/hr or more. Performance tests are normally done at maximum, minimum and at least one intermediate point.
AAMI thought the same, and introduced the term “maximum selectable rate”, and added another column to Table 102, requiring tests at the maximum rate for normal volumetric and other types of pumps with continuous flow (excluding ambulatory pumps).
Sounds good? Not in practice. The tests in the standard cover three key aspects - start up profile, short to medium term variability, and long term accuracy. Studying all three parameters at all four points (lowest selectable, minimum, intermediate and maximum selectable, i.e. 0.1, 1.0, 25, 999mL/hr) is something of a nightmare to test. Why? Because range from represents 4 orders of magnitude. You can’t test that range with a single set up, you need at least two and possibly three precision balances with different ranges.
IEC2 again started to follow AAMI, adding the new defined term “maximum selectable rate”. But again, that’s pretty much as far as it got. It was never added to Table 201.102. Fortunately, unlike the other stuff ups, this one is benign in that it does not leave the test engineer confused or force any unrealistic test. It just defines a term that is never used.
So the real world solution? A rate of 25ml/hr seems a good rate to inspect the device’s flow rate graph, trumpet curves and long term accuracy. As a pump gets faster, start up delays and transient effects will become less of an issue. This suggests that for higher speeds, the only concern is the long term flow rate accuracy. This can be assessed using fairly simple tests that use the same precision balance (typically 220g, 0.1mg resolution). For example, an infusion pump with 1000mL/hr range could be tested at 100, 500 and 1000mL/hr by measuring the time taken to deliver 100mL. The flow rate accuracy and errors can be easily be determined from this test without needing multiple balances.
IEC1 used a sample interval of 30s for general purpose pumps and 15min for ambulatory, both of which seem reasonable in practice.
AAMI changed this to 10s for general purpose pumps. The reason was probably to allow the true profile to be viewed at the maximum selectable rate, such as 999mL/hr. It also allows for a smoother looking trumpet curve (see below).
While that is reasonable, the drawback is that for low flow rates, the background “noise” such as resolution of the balance will appear three times larger, which is unrelated to the infusion pump’s performance. For example, a typical 220g balance with a 0.1mg resolution has a rounding error “noise” of 1.2% at rates of 1mL/hr sampled at 30s. This noise has nothing to do with the pump’s performance, it’s just the limits of measurement of the set up. If the sample interval is reduced to 10s, that noise becomes 3.6%, which is highly visible in a flow rate graph, and has no clinical relevance.
Needless to say, this is another AAMI deviation that seems to have been ignored in practice.
IEC2 again appears to have started to follow AAMI, and got as far as deleting the reference to a 30s interval (0.5 min). But, that’s as far as they got - the sentence now reads “Set the the sample interval S”, which implies that S is somehow defined elsewhere. Newbie test engineers unaware of the history could spend hours searching the standard to try and find where or how S is determined. A clue that it is still 0.5 min can be found in Equation (6) which defines j and k as 240 and 120, which only makes sense if S = 0.5 min. With the history thought it seems clear that there was an intention to follow AAMI, but cooler heads may have prevailed and they just forgot to re-insert the original 0.5min.
For general pumps, IEC1 requires the analysis to be performed at 2, 5, 11, 19 and 31 min (with similar points for ambulatory adjusted for the slower sample rate). That requires some 469 observation windows to be calculated, with 10 graph points.
The AAMI puts this on steroids: first it introduces a 10s sample interval, next it expands the range down to 1min, and finally requires the calculation to be made for all available observation windows. That results in some 47965 window calculations with 180 graph points
That said, the AAMI approach is not unreasonable: with software it is easy to do. It is possible to adopt a 10s sample interval from which a smooth trumpet curve is created (from 1 to 31min as suggested). It is unclear why the IEC version uses only 2, 5, 11, 19 and 31min, and it is possible that some pumps may have problem intervals hidden by the IEC approach.
The flow rate curve can also be made smoother using the 10s data, by calculating the flow every 10s using a 30s interval. Conceptually
In IEC1 and AAMI, there are 5 pump types in the definition of an infusion pump. The types (Type 1, Type 2 etc) are then used to determine the required tests.
In IEC2, the pump types have been relegated to a note (which is informative only), and only 4 types are shown: Type 4 (combination pumps) were deleted, and profile pumps, formally Type 5 are now referred to as Type 4.
Unfortunately this again seems to be a case of something half implemented. While the definition was updated the remainder of the standard continues to refer to Types 1 to Type 5 as before, and refers to them as if they are defined terms, not an informative note. Again, a newbie trying to read the standard would be confused without the history.
Experience from testing syringe pumps indicates there is a huge amount of variability to the test results from one syringe to another, and from one brand to another, due to variations in “stiction”: the stop start motion of plunger. And even for peristaltic type pumps, the accuracy is likely to be influenced by the tube’s inner diameter, which again is expected to be difficult to control in practice, leading to variable from different standards.
Nevertheless, the standard only requires the results from a single “type test” to be put in the operation manual, leading to the question - what data should I put in? One possibility is to run a bunch of tests and then select either the worst case, the typical case or the best case depending on whether you are overly cautious, somewhat reasonable or a corporate snake. As it turns out, competitive pressures as such that manufacturers use the “best of the best” data in the operation manuals, which is totally unrepresentative of the real world. Their argument is that - since everybody does it, everybody … well … has to do it. The responsible manufacturer that does honestly declare their performance won’t be able to sell their product. Which is actually a fair argument, and precisely why we need standards to force responsible approach under a common set of rules that everybody has to follow.
In IEC1, this issue is discussed in Appendix AA.4, but it is an odd discussion since it seems disconnected with normative requirements in the standard: it concludes that for example, a test on one sample is not enough, yet the normative parts only requires one sample.
IEC2 includes the same informative annex, but at least they preface it with a statement to the effect that this has nothing to do with the normative part of the standard and is there just for future consideration. Which is odd since there already had 14 years for consideration and really, it’s not that complicated and also a critical issue - what’s the point in running all these test if data in the operation manuals is effectively junk?
In Europe, a standard must reasonably match the essential requirements; in this case a key requirement is 12.8.2: Devices for supplying the patient with … substances must be designed and constructed in such a way that the flow-rate can be set and maintained accurately enough to guarantee the safety of the patient … . The data generated in IEC 60601-2-24 testing obviously forms an important part of determining if this requirement is met. Selecting the “best of the best” does not answer this essential requirement.
One might wonder why the IEC 60601-2-24:2012 (EN 60601-2-24:2015) is yet to be adopted in Europe. In fact there are quite a few particular standards in the IEC 60601 series which have yet to be adopted in Europe. One suspects that the regulators are starting to having a close look at these standards and questioning whether they do, indeed, address the essential requirements.
Moreover, It may be that the IEC committees are simple ill equipped or structurally unsound to provide standards that genuinely cover reasonable requirements and are checked for obvious errors . The current model that relies on volunteers, heavy presence of manufacturers, and an attitude of “all care, no responsibility”, and the use of national committees to provide feedback is simply not working. We need designated agencies to sit down and actually try to implement the standard at the DIS or FDIS stage to check for errors and plausibility.
Something needs to change.