Filter Configuration

Configuring tag filters is introduced in Setting up Tag Filters. The following information describes the filter configuration process in detail.

Create tag range and filter

Run Location System Config. Go to the Set up tag filters tab.

Add a new filter and give it an appropriate name.

You can configure different filter settings for:

  • A tag range, by specifying the filter to be assigned to a range of tag IDs.
  • A tag type, by specifying the filter to be assigned to a tag type (this takes precedence over a tag range).
  • A particular tag, by specifying the filter to be assigned to a specific tag ID (this takes precedence over a tag type).

Add a new tag range. The default range is 00:11:CE:00:00:00:00:00 to 00:11:CE:00:FF:FF:FF:FF, covering all possible tag IDs. If you have different sets of tags, create a tag range for each set of tags with consecutive tag IDs. Then choose the appropriate filter for each tag range.

Add a tag type and specify the name of the tag type, and then choose the appropriate filter for each tag type.

If there are any individual tags which should use a particular filter, select "Add a new tag" for each one, enter the tag ID and choose the appropriate filter.

Note that DIMENSION4 tag IDs are 8 bytes, but the top four bytes are usually the same so some tools only show the bottom four bytes. But for defining tag ranges, you have to specify all eight bytes.

Set speed and height for this installation type

There are several physical filter parameters which should be specified for all filters. They are:

  • Horizontal Speed
  • Minimum Height
  • Default Height
  • Maximum Height

The heights don't need to be too accurate, as long as they don't contradict the actual heights the tags will usually be used at. For instance, the default values are 0.5 m (knee height), 1.0 m (waist height) and 1.5 m (shoulder height), which usually work well for tags on tools carried by a person.

Heights are specified relative to the floor of the clipping region of whichever sensor group is currently detecting the tag. So the same filter parameters work even if a tag is used upstairs and downstairs, as long as the sensor heights and sensor group clipping region are set to the correct absolute heights.

Your best estimate for the speed the tags will usually move at is a good starting point for the horizontal speed parameter, but small changes to the value can sometimes have large effects, so if the site needs reliable locations, you may have to experiment to see which values work best. A common value is 2.5 m/s (about 5½ mph or 9 kph, which is a slow jogging speed).

For all these values the most important consideration is to set them to be as true to your actual situation as you can.

Set motion model and any special flags for this type of site

Motion model should be set to 2 ("Stationary") for most use cases.

The advanced flags should be configured with the settings discovered to work best at previous similar sites. For instance:

  • For a demonstration system (walking speed in office without excessive metal reflectors)
    • Use default settings
  • For assembly line, car and tool tags
    • Use most recent recommended settings. At the time of writing those are:
      • Set "Limits/Minimum Reset Confidence" to "3"

Ident sensors don't use a user-configured filter. Instead they are designed to produce a location when, and only when, an angle from the ident sensor intersects the floor of the clipping region of its ident group.

Evaluate locations, and repeat as necessary

For many simple situations, setting the parameters as described above is sufficient. For sites with challenging environments or with challenging requirements on location responsiveness you can experiment to get better locations, using command-line tools to test which settings work best.

The command-line tools work by pulling data back from the logged data that is managed by the logging server, and converting it into an interchange format: a .ubievents file.  There is a suite of tools available for analyzing recorded events; these operate on the interchange format. The Location System Config application can read and write events to this format using the Import/Export buttons. The tools include:

ubisense_ls_event_to_file.exe  

Input: From and To times, logging API

Output: Event interchange file

This program supports exporting of data from the stored events system to an events interchange file.

ubisense_ls_file_analyser.exe

Input: Event interchange file

Output: Text analysis summary

This program reads events from an interchange file and generates a set of statistics on a per-tag basis, including jumps, outages and sighting rate.

ubisense_ls_event_analyser.exe

Input: From and To times, logging API

Output: Text analysis summary

This program is like the file analyser above, but instead of reading events from a file it reads them using the logging API.

ubisense_ls_refilter.exe       

Input: Event interchange file, current filter settings in dataset

Output: Event interchange file

This program reads measurements from an interchange file and uses the existing filter settings in your dataset to refilter the events. Note that for any refiltered period the start may appear to be slightly different as the current filter state is not recorded.

ubisense_ls_file_converter.exe

Input: Event interchange file

Output: Event interchange file

This program rewrites an event interchange file with user-specified modifications. Currently the supported modifications are: choosing which sensors' readings to keep, choosing which tags to keep and whether to divide the tag rates and truncating the event file by event index from the start or end.

For each of these tools a detailed help message is generated by typing <toolname> --help on the command line.

Although it is not an analysis tool, the command line program ubisense_ls_filter_admin can be useful for an analysis/refilter workflow as it can be used to set and list filter parameters from a script.

A simple usage scenario for the tools at a small installation would be something like this:

  1. Create a sample .ubievents file to test by retrieving events in Location System Config in the Review location events tab and selecting "export", or by using the command-line tool:

    ubisense_ls_event_to_file.exe "Cell name" YY-MM-DD-hh-mm-ss YY-MMDD-hh-mm-ss [UTC+hh:mm] filename.ubievents
  2. Run ubisense_ls_file_analyser.exe with a command line like:

    ubisense_ls_file_analyser.exe summary filename.ubievents

    It will produce a set of statistics describing possible problem events.

    You can also use ubisense_ls_event_analyser.exe to produces statistics on events from the log server without creating an intermediate file in step 1.

  3. If the statistics already look acceptable, you can stop. Else choose some new filter parameters in Location System Config, or by creating a text file listing the parameters and using ubisense_ls_filter_admin.exe.

    Then refilter the events using a command line like:

    ubisense_ls_refilter.exe filename.ubievents refiltered.ubievents

    Repeat the previous step to see if the statistics have got better or worse. Examine the events file in Location System Config to be sure you've not introduced any new errors.

  4. It is common to set up a series of scripts to automate this process. For instance, having a collection of input .ubievents files for different tags on different days, and producing output .ubievents files and statistics on each of them in one step. For instance, creating the alternative filter parameter configurations as text files and using ubisense_ls_filter_admin.exe in a script to automatically apply each of them in turn.

For larger installations that have an ongoing installation process it is common to schedule a script to do to a batch analysis on a day's worth of data to run each night. These scripts usually need to be customized for each installation, but there are samples available.

Filter Parameters

Basic Parameters

Anyone should be able to configure the basic parameters with knowledge of how it will be used, without an extensive understanding of how DIMENSION4 filtering works.

Motion Model

The motion model is used to extrapolate the tag's motion.

  1. = "Planar". Tag position moves in a fixed-height horizontal plane. Used internally by ident sensors.
  2. = "Stationary". Tag position can move horizontally and vertically, constrained between minimum height and maximum height. Use this for all normal operation.

Advanced: If you want a fixed height filter, it's almost always better to configure minimum and maximum height parameters instead of using planar, so minor elevation errors don't make the tag jump large horizontal distances.

Advanced: A zero-value is used internally to indicate an initialized filter state. Values 3 onward are reserved for constant speed models and multi tag models.

Horizontal Speed (m/s)

The maximum speed the tag can move at before the calculated position starts to lag. Usually set this to slightly higher than the fastest speed the tag will ever really go.

If there are short bursts of faster movement the calculated position will usually catch up very quickly even if it lags. It doesn't cause any problems if the tag moves slower than this. But if it's set to a very high value, the reported location will jitter a lot, and jump too quickly to respond to new measurements in the wrong place.

Usually set this value to the best value found at previous similar sites (e.g. previous assembly lines). But if the new site is different, it may be worth testing different values to see which works best.

Advanced: This parameter controls how fast the variance increases in between measurements. The conversion from speed to variance is only approximate, it's necessary to experiment to find which value works in real operation.

Vertical Speed (m/s)

As "Horizontal Speed".

Vertical motion is usually not as important and not as fast so you usually don't need to change to change this.

Minimum Height (m)

Prevent the calculated location going below this height.

Used to produce better locations when there are no good elevation measurements. Usually set to the nearest half-metre. It doesn't need to be more precise than that.

Advanced: Setting maximum height and minimum height to the same value is sometimes useful but it's usually better to have a narrow range of height.

Advanced: If you want to allow the tag to move above or below a height but not report locations when it does, use a clipping region with those heights.

Maximum Height (m)

Prevent the calculated location going above this height.

Used to produce better locations when there are no good elevation measurements. Usually set to the nearest half-metre. It doesn't need to be more precise than that.

Advanced: Setting maximum height and minimum height to the same value is sometimes useful but it's usually better to have a narrow range of height.

Advanced: If you want to allow the tag to move above or below a height but not report locations when it does, use a clipping region with those heights.

Default Height (m)

Use this height in initialization if the true height can't be determined. Also used in some other circumstances, for instance, in ident sensors when there's no other way to determine the height.

If you're not sure what value to use, using half-way between Minimum Height and Maximum Height usually works reasonably well.

Limits

Limits includes various internal thresholds and limits. These should usually be set to the same values for several sites of the same sort.

Initial Variance

The filter is usually initialized to this variance (see also nonconfident variance multiplier).

The exact value shouldn't make a difference.

Usually slightly higher or slightly lower than report variance.

Used with good support count to trade-off between "after reset, how soon we get a report" and "after reset, how likely are we to get a false positive location message" .

Initial Nonconfident Variance Multiplier

If Minimum Reset Confidence allows the filter to reset to positions based on only one or two angles, the initial variance is multiplied by this number so the filter doesn't report a location immediately.

If Minimum Reset Confidence is 2 or 3, it doesn't make a difference.

Usually set to 3, so filter reports locations only after several further supporting readings. Set to 1 to report locations equally soon after any sort of reset. There is no reason to set it to any value below 1.

Delta-T Limit (seconds)

If the filter goes this amount of time without reporting a valid location, attempt to reset.

Report Variance

Filter reports the current location when the variance is less than this limit.

It needs to be lower than the variance of locations when the filter is getting good measurements so the filter ever reports locations. Other than that, it controls how quickly a filter starts reporting locations, and how quickly it stops reporting locations when it doesn't receive enough measurements.

You may have to adjust this if you change horizontal speed significantly.

Max Variance Before Reset

Filter attempts to reset when the variance exceeds this value.

Usually because sensor has stopped receiving measurements, or because the tag has moved quickly and the new measurements are being rejected as too far from the last-known location.

Low Support Count (Seconds)

If over half of the measurements made disagree with the filter's current location, or there are too few measurements, the currently reported location may be incorrect. The location has "low support" from the measurements.

If there are no locations with good support for this many seconds, force a reset.

This is the usual way the filter resets after a period with no locations.

Good Support Count Limit

After the filter is reset, wait for this many consecutive events with good support before producing a location, regardless of the variance.

Usually 0, 1, or 2.

In theory, false positives can be prevented by correct values for initial variance and report variance, but keeping them adjusted is complicated. Good support count limit removes most false positives in situations where waiting for 2 updates is acceptable.

Some configurations never produce good support (e.g. a group with a single sensor). Those will only produce a location if this parameter and Nonconfident Additional Good Support Count Limit are set to 0.

Nonconfident Additional Good Support Count Limit

The filter waits this many additional consecutive events on top of Good Support Count Limit if the reset position did not count as confident.

This means spurious resets will usually not report a location.

Default value is 2, this is usually sufficient to prevent spurious events.

Some configurations never produce good support (e.g. a group with a single sensor). Those will only produce a location if this parameter and Good Support Count Limit are both set to 0.

REMOVED (Minimum Measurement Power)

This parameter has been removed.

There are two other parameters you may wish to use.

Sensor/Sample Processing/Peak Detect Threshold tells an individual sensor only to make measurements that exceed this power threashold.

Group/Measurements/Power Acceptance Threshold tells a group only to process sets of measurements where one or more measurements exceeds this value.

Minimum Reset Confidence

Allow resets with this level of confidence:

-1 = No result, use position "near" sensors

0 = No agreement, use best available measurements even if far apart, or intersection of single measurement with default height

1 = Unique position specified

2 = Corroborated position

3 = Doubly corroborated position

Maximum Angle Vnorm

The Vnorm measures how far an angle is from the expected location. If it is larger than this value, the measurement is rejected as a probable-reflection.

Maximum TDOA Vnorm

As "Maximum Angle Vnorm" for TDOA measurements.

Continue Reporting Last Known Position/Max ms Between Accepted Measurements

When the location becomes invalid, continue reporting the last known location as long as there are some measurements which are accepted by that position.

For instance, sometimes a vehicle is partially obscured by a pillar, and there are periodic measurements, enough that it is obvious that the last location is probably valid if you know the vehicles are usually stationary, but not enough to generate a location itself. This setting tells the filter to keep that location alive.

Default is 0 (equivalent to off).

Continue Reporting Last Known Position/     Effective Variance

Variance used to determine if a measurement is accepted in Continue Reporting Last Known Position mode.

Flags

Flags enable and disable behavior which is needed in specific situations.

Always Require Three Degrees Of Freedom For Support

Should always be "True".

Only applies to planar motion model and other non-3D motion models. Updates only one angle have two degrees of freedom and theoretically are enough to determine a position, but usually represent bad data, so count them as updates with low support.

Low Support Counts Measurements Not Degrees Of Freedom

Should always be "True"

When determining if an update has low support, consider "half the measurements are accepted" don't count azimuths and elevations separately.

Increase Angle Sigma For Non-bore Angles

Should always be "False".

Azimuth and elevation measurements are less accurate for angles at extreme sideways edges of the range. The measured value should therefore be assigned a higher variance and hence a smaller effect on the current position.

However, the implementation needs to be fixed before it can be used. It should only affect the information filter, not the code that tests angles for valid (i.e. reject reflections), else extreme angles are simply never rejected.

Reject Angle Measurement If TDOA Rejected

Should always be "True"

If a TDOA measurement is clearly incorrect, it must be from a reflection, and the angle must also be meaningless, so discard everything else from that sensor.

The reverse is not true: walls can scatter radio signals a little bit, meaning the angle is somewhat wrong, but the TDOA is nearly accurate.

Fast Reset With Confident Result

Reset immediately if there's a sufficiently good position to reset to.

Should be turned off for assembly lines to prevent false positives, but may track more consistently in some noisy environments.

Clip Height

Restrict vertical position between Minimum Height and Maximum Height.

This should usually be on. Turn it off to allow locations with completely unbounded heights.

Prefer Angles

Use angles in preference to TDOAs.

It is implemented by using a multiplier of the smallest angle vnorm as the cut-off for accepted TDOA vnorms.

Give slightly worse locations, but gives better results for Left/Right determination inside a car in some environments.

Continue After Failed Variance Reset

Should usually be set to "False".

If the variance exceeds Max Variance Before Reset, the filter resets if possible. If not, the possible position has such a wide range it provides no useful input into the next position.

This flag instructs the filter to abandon the position in that situation and require a reset from scratch. If it is not set, the position will eventually converge on what measurements there are, but not necessarily the best ones.

This flag used to be combined with "Continue After Failed LSC Reset". Now they are separate there is usually no reason to change either.

Continue After Failed LSC Reset

Should usually be set to "True".

If the number of seconds since the last location with good support exceeds "Low Support Count (Seconds)" the filter resets if possible.

If it can't reset, but then gets good measurements again, this flag instructs it to resume reporting a location.

If the variance is still below Max Variance Before Reset, this is usually better than requiring a reset from scratch. 

Clustering Require Tdoa For Corroboration

Should be set to "True".

Reset confidence 3 requires measurements from two sensors plus an angle from a third sensor for corroboration. This flag requires a corroborating angle from a sensor where the TDOA also agrees.

Customization

Customization parameters control other parts of the filter that may be adjusted, especially in reset.

Reset Confidence From TDOA Triple        

Default 2. If this is equal or more than Minimum Reset Variance, the filter  can reset from three TDOAs but no angles. TDOAs are usually more accurate. But some sites have an ongoing problem with reflections and prefer to turn off all-TDOA resets by setting this to 0 or below.

It also controls which measurements are used to choose a pivot.

Testing

Testing parameters should not be used in normal operation, but are useful to diagnose certain problems.

Bypass                         

Skip filter code entirely.

Always Succeed             

Ignore validity and always report location as valid instead.

Logging Level

Not used

Ignore Angles

Ignore all angle data for all purposes. Used to test performance and test filtering of TDOAs.

The data will still be present in the event message to the platform and the used/not used coloring in LSC may or may not be correct.

Ignore TDOAs

Ignore all TDOA data for all purposes. Used to test performance and test filtering of angles.

The data will still be present in the event message to the platform and the used/not used coloring in LSC may or may not be correct.

Super-Generous

Always report a location, as long as the filter is initialized. Will sometimes be out-of-date or incorrect.

Stateless

Ignore previous state and report the location most consistent with measurements from the current update.

Performance

Performance parameters enable or disable several optimizations in the filter code. Currently the bottleneck is elsewhere in the sensor code, so these changes make no difference to performance. 

Fast Clustering

Use the first acceptable combination of measurements when choosing which TDOAs to apply, don't try all and choose the best.

This makes no difference to performance. It usually shouldn't be used except in testing.

No Clustering Round Robin

Skip clustering entirely and apply TDOAs from sensors in round-robin, regardless whether those TDOAs are correct.

This makes no difference to performance, It usually shouldn't be used except in testing.

Maximum measurements per update

Consider measurements from this many sensors, ignore the rest. This is mostly obsolete by a fixed limit on the number of measurements in the sensor code.

A lower value would allow more updates per second, but if it's too low, the filter couldn't produce good locations.

0 means "no limit". Values below 5 will usually give worse locations. 5 is usually acceptable, but it is different at different sites. 10 should always be enough.

Group Parameters

Three parameters are a property of the sensor group, not the filter, when they are more closely associated with the environment a group is in, than the characteristics of the person or object the tag is on.

The most common use is when there are sensors inside and outside a building, but the locations should be more forgiving in groups outside because the environment is less noisy, but there are much larger distances between the sensor and the tag.

They can be configured in the Group/Filter section of the Change all parameters tab of Location System Config.

Variance Multiplier

Multiply all variance parameters (Initial Variance, Max Variance Before Reset, Report Variance) by this value. Default is 1.0 which has no effect. A higher value makes the filter more forgiving.

Allowable Reset Confidence

As " Minimum Reset Confidence " but if this value is lower, it takes precedence. so it can make the filter reset more generous in some groups, (but not more strict.)

Use this for a group which should reset with a smaller number of measurements e.g. because there are fewer sensors, an environment with more obstructions or fewer reflections, or the tag is a long way from the sensors. But see also Scaling Multiplier.

Scaling Multiplier

This governs the distance between measurements which count as agreeing on a filter reset position.

The default is 1.0. This is best for most groups.

Groups where all the sensors are a long distance apart may work better with a higher value. For instance, if the sensors are 100m apart, and this parameter is set to 5.0, they will reset as easily as if they were 20m apart. Remember that this may allow a reset with two angles which are physically several meters from the proposed reset position. You should generally try several values between 1.0 and the highest value you expect to need and see which works best.

It doesn't affect TDOAs because TDOAs are do not get proportionally more inaccurate with distance.

Concepts

Filter state

The filter has two modes of operation. To begin with it waits for an event with several measurements which all agree on a single position. This is called a reset event. The combinations of measurements are described below.

Until an event with measurements that agree is found, the filter produces no locations. If the measurements are bad, e.g. if the sensors are uncalibrated, the filter may never produce any locations. Sometimes this is what you want, if bad locations are worse than no locations. If you want to produce some locations immediately, you can set "minimum reset confidence" to 0 or even -1 to ensure it produces locations as soon as it gets some measurement, even a bad one.

Once the filter has a location, it keeps updating that location with new measurements. It tracks the location and variance. The variance is used to determine which measurements are accepted.

Measurements which are too far from the existing location are assumed to be reflections are rejected. Measurements which are sufficiently close are accepted, and the location is updated.

The location does not just instantly become the best fit for new locations. Instead, the amount it moves depends on the variance. If the variance is large, i.e. there haven't been any measurements for a while, the previous position is given very low weight. If the variance is very small, e.g. there is a high hz tag visible from many sensors, only measurements very close to the location are considered.

If the variance exceeds the Maximum Variance Before Reset parameter, or there is continued low support, or there is no location for Dt Limit seconds, the filter stops reporting locations and tries again to reset.

Clustering Methods and Confidence

When the filter evaluates a reset position, it considers several different possible combination of angles, called clustering methods. It assigns a confidence value to a combination according to how reliable it is. The specific combinations may be added to over time, but at the time of writing they are:

Reset location

Confidence value

Angles + TDOA from two sensors, plus corroborating angle 

3

Angles + TDOA from two sensors

2

Three TDOAs which agree at a point

2 (by default)

Along angle from single sensor, at height equal to Default Height parameter

0

Average of multiple sensor positions

-1

1m in front of single sensor

-1

The confidence values represent how reliable a particular combination of measurements is.

Confidence value

Description

-1

There is no location from the measurements at all.

This only occurs when no sensor produces an angle, and the TDOAs are insufficient to identify a location. There is often TDOA at all, as when only a single sensor produces a time of arrival or when multiple sensors from different timing sources produce times of arrival. However, even if there are one or two TDOAs, that is not enough to uniquely identify a location.

This usually indicates that there is no good tracking available, or the sensors have not been configured yet.

But sometimes simply knowing which sensors saw the tag gives some information about where it might be. If you are not concerned with some initial bad locations and allow the filter to reset with this confidence, it is often resets close enough to the tag that it it may converge to an accurate location over time.

0

The measurements can identify a location, but there is no way to tell if it is better or worse than any other location.

If there are no reflections, this position is almost certainly correct. But if there are multiple measurements, there is no way to choose which position is correct.

1

Not currently used

2

This requires two separate measurements to agree when reflections or other bad measurements would have no reason to do so.

For instance, two angles + a TDOA. Any two angles will usually not intersect at all. If they do, that is one point of correspondence, suggesting they are more likely to come from a true location . If two angles intersect by chance, there is no reason that the TDOA between those sensors should also intersect the same point, unless the location is true. So if it does, that's another point of correspondence.

Two points of correspondence is enough that this does not usually occur by chance, so is usually a true location.

3

As 2, but requiring an additional point of correspondence.

Currently the only mode which produces confidence 3 is "two angles plus tdoa, plus an additional angle". The additional angle is accepted as corroboration without a TDOA if it comes from a sensor with a different timing source, but both are required to agree if the TDOA exists (if Require TDOA for Corroboration is set).

Some of this behavior can be customized. Some sites have very bad TDOAs, so it is possible to set the "Reset Confidence From Tdoa Triple" in order to choose whether the confidence reported from a location from three TDOAs is 2, or lower, or higher. If this is lower than "Minimum Reset Confidence", three TDOAs will not be used to reset the filter. And if it is lower than 2, they will usually not be used for clustering if another clustering method is possible.

The distance between measurements in order for them to count as agreeing on a point can be customized by setting the "Scaling Multiplier" of a group.

Reporting filter bugs effectively

If you encounter problems with the filter, it is usually due to the filter behaving as expected in a situation with insufficiently good measurements, but sometimes there is an actual bug, or a feature which should exist and doesn't.

Explain clearly, with specifics, what happened, and how it different from what should have happened. Ideally you should include:

  • A .ubievents file showing the incident
  • Say which event index the problem is at
  • Describe where the tag actually is at the time and what should have happened. For example, "there are good measurements and I expected to get a location". Don't assume this is obvious
  • Check which is the first event which seems to be wrong, and give the event number
  • How urgent is this problem? Do you need help trying to meet an immediate need? Or are you suggesting something which may be fixed in a future DIMENSION4 release?

You may also provide a dataset, but that is much less important for filter problems that the above information.

Consider what actually happened, not only the particular event produced a problem at a higher level. Usually the key is the most recent reset event. Did the filter reset when it shouldn't? Not reset when it should? Remember, the filter often waits a couple of events after a reset before reporting a location, so look at the location in the events tab, not only whether there's a red dot or not. If the filter location is already wrong, one good set of measurements doesn't fix everything, 

A good report might look something like this:

See the attached event file. At event 334, the tag is on the left side of the assembly line, but the filter resets way off the line to the right. It doesn't produce a location for several more events, but at event 371, it produces a location close to the initial reset position. This causes a problem because X.

I don't understand why it resets in that place. Can you help me understand why that happens? Is it a bug, or a normal part of filter behavior I don't understand? Are there any changes to the filter parameter which would let us avoid this problem? If not, is there any code fix which could be made for the next release?