Advanced Filters

This section covers the following:

• Overview
• Displaying Matches While Using Advanced Filters
• Using The Advanced Filter Manager
• Writing Expressions
• Expressions That Check For Data Types
  • Data Type Presence Check
  • Data Type Count Comparison Operators
  • Data Type Function Check
  • Data Type Sets
• Logical and Grouping Operators
  • Logical Operators
  • Grouping Operators
• Remediating Matches While Using Advanced Filters

Overview

There are situations where a certain combination of data types can provide more meaningful insight for matches found during the scans. Specifically, during analysis of scan results, such combinations can be helpful when attempting to eliminate false positive matches while at the same time homing in on positive matches with greater confidence.

For example, consider a situation where a scanned location A has matches for phone numbers, scanned location B has matches for email addresses, while scanned location C has matches for both email addresses, and phone numbers.

In the example above, it is more likely that location C would actually have Personally Identifiable Information (PII) targeted at an individual compared to locations A and B alone. This is because location C contains two items of data that can be related to an individual. We can use Advanced Filters to display such locations.

Displaying Matches While Using Advanced Filters

To view match locations that fulfill the conditions defined in an Advanced Filter:

1. Log in to the ER2 Web Console.
2. Go to Investigate.
3. In the Filter Locations By or Filter panel, click on Advanced Filters.
4. Select one or more Advanced Filter rules to display specific match locations.

Using The Advanced Filter Manager

Use the Advanced Filter Manager to:

1. Add an Advanced Filter
2. Update an Advanced Filter
3. Delete an Advanced Filter

Add an Advanced Filter

1. Log in to the ER2 Web Console.
2. Go to Investigate.
3. In the Filter Locations By or Filter panel, click on Advanced Filters.
4. Click on Manage to open the Advanced Filter Manager.
5. In the Filter name field, provide a meaningful label for the Advanced Filter.
6. In the Filter expression panel, define expressions for the Advanced Filter. See Writing Expressions for more information.
7. Click Save Changes. The newly created filter will be added to the list on the left.

Update an Advanced Filter

1. Log in to the ER2 Web Console.
2. Go to Investigate.
3. In the Filter Locations By or Filter panel, click on Advanced Filters.
4. Click on Manage to open the Advanced Filter Manager.
5. Select an Advanced Filter from the list.
6. Edit the filter name or expression for the Advanced Filter. See Writing Expressions for more information.
7. Click Save Changes.

Delete an Advanced Filter

1. Log in to the ER2 Web Console.
2. Go to Investigate.
3. In the Filter Locations By or Filter panel, click on Advanced Filters.
4. Click on Manage to open the Advanced Filter Manager.
5. Select an Advanced Filter from the list.
6. Click the trash bin icon next to the filter name.
7. Click Yes to delete the Advanced Filter.

Writing Expressions

Each Advanced Filter is defined using one or more expressions which are entered in the editor panel of the Advanced Filter Manager. There are a few basic rules to follow when writing expressions:

• An expression consists of one or more data type names combined with operators or functions, and is terminated by a new line.

  1	[Visa] and [Mastercard]
  2	[Passport Number]

  In the example above, line 1 and line 2 are evaluated as separate expressions and is equivalent to defining two separate filters with one line each. New line separators are interpreted as OR statements. See Logical Operators for more information.

• Each expression evaluates to either a true or false value. If an expression in a filter evaluates to true for a given match location then that match location is displayed.

• Expressions are evaluated in order of occurrence. When an expression is evaluated and returns a positive result (true), the match location is marked for display and no further expressions are evaluated for that filter.

  1	[United States Social Security Number]
  2	[United States Telephone Number] AND [Personal Names (English)]

  In the example above, a given match location is first checked for the presence of a United States Social Security Number. If a United States Social Security Number is found, line 1 evaluates to true and subsequent lines are skipped. If no United States Social Security Number match is found, line 1 evaluates to false and the match location is then checked for a combined presence of United States Telephone Number and Personal Names (English) matches.

• For readability, a single expression can be split across multiple lines by ending a line with a backslash \ character.

  1	[Visa] AND \
  2	[Mastercard] OR \
  3	[Discover]

• Comments are marked by a hash # character and extend to the end of the line. Comments can start at the beginning or in the middle of a line, and can also appear after a line split. All comments are ignored by the Advanced Filters during evaluation.

  1	# This is a comment
  2	[Visa] AND \ # Look for Visa
  3	[Mastercard] OR \ # Look for Mastercard
  4	[Discover] # Look for Discover

• White spaces are optional when defining expressions unless they are required to separate keywords or literals.

  1	[Visa] AND MATCH(2, [Login credentials], [IP Address], [Email addresses])
  2	# line 1 can also be written as line 3
  3	[ Visa ] AND MATCH(2, [ Login credentials ], [ IP Address ], [ Email addresses ])

Expressions That Check For Data Types

The simplest Advanced Filter expression is one that checks for the presence of a specific data type match in a scanned location. This is called a Data Type Presence Check.

You can find a full list of built-in data types and their names when you Add a Data Type Profile. These data type names:

• Are case sensitive.
• Must be enclosed in square brackets [].
• Have robust and relaxed variants. If not specified, the relaxed mode is used. For example, the Belgian eID data type has the Belgian eID (robust) and Belgian eID (relaxed) variants. ER2 defaults to using Belgian eID (relaxed) if you don’t specify the variant to use.

The Advanced Filter editor has an AutoComplete feature that helps you with data type names. To use AutoComplete, press the [ key and start typing the data type name to include in your expression.

The AutoComplete feature only lists the data types that have matches for your Target, but you can still define data type names that have not matched in your Advanced Filter expressions.

Data Type Presence Check

Checks for the presence of a data type in a match location.

Syntax

[<Data Type>]

Example 1

Example 1 lists match locations that contain at least one Personal Names (English) match.

Example 2

Example 2 lists match locations that are not Visa data type matches.

Data Type Count Comparison Operators

Use comparison operators to determine if the match count for a data type meets a specific criteria.

Syntax

[<Data Type>] <operator> n

n is any positive integer, e.g. 0, 1, 2, ..., n.

Operators

Example 3

Example 3 lists match locations that contain at least two Personal Names (English) matches.

Example 4

Example 4 lists match locations that contain less than three Login credentials matches or contains no Email addresses.

Data Type Function Check

MATCH function checks for the presence of n unique data types from a list of provided data types, where the number of provided data types has to be greater or equal to n.

Syntax

MATCH(n, [<Data Type 1>], [<Data Type 2>], ..., [<Data Type N>])

n is any positive integer, e.g. 0, 1, 2, ..., n.

Example 5

Example 5 checks match locations for Visa, Mastercard, Troy, and Discover matches, and only lists a match location if it contains at least two (n=2) of the four data types specified. In this example:

• A match location that contains one Visa match and one Troy match will be listed.
• A match location that contains Mastercard matches but does not contain any Visa, Troy or Discover matches will not be listed.

Data Type Sets

Use SET to define a collection of data types that can be referenced from the MATCH function.

Syntax

SET <set identifier> ([<Data Type 1>], [<Data Type 2>], ..., [<Data Type N>])

When defining a SET, follow these rules:

• A SET definition is a standalone expression and cannot be combined with any other statements in the same expression.
• SET must be defined before any expression that references it.
• SET identifiers are case sensitive.

Example 6

Example 6 defines a set of data types named CHD_Data in line 1. It then uses a MATCH function call to check scanned locations for the presence of matches for the data types specified in the CHD_Data set. Any scanned location that contains at least two of the data types specified in the CHD_Data set will be returned as a matched location. The following locations will be returned by the filter. In this example:

• A match location that contains one Visa match and one Troy match will be listed.
• A match location that contains one Mastercard match but does not contain any Visa, Troy or Discover matches will not be listed.
• A match location that contains two Mastercard matches but does not contain any Visa, Troy or Discover matches will not be listed.

Logical and Grouping Operators

Use logical and grouping operators to write more complex expressions. Operator precedence and order of evaluation for these operators is similar to operator precedence in most other programming languages. When there are several operators of equal precedence on the same level, the expression is then evaluated based on operator associativity.

Logical Operators

You can use the logical operators AND, OR and NOT in Advanced Filter expressions. Logical operators are not case sensitive.

Operators

Example 7

In Example 7, line 1 lists match locations that do not contain Visa matches.
Line 2 lists match locations that contain at least one Login credentials match and at least one Email addresses match.

Example 8

In Example 8, line 1 lists match locations that contain at least one Australian Mailing Address match or at least one Australian Telephone Number match.

Instead of writing a chain of OR operators, you can write a series of data type presence checks to keep your expression readable. For example, Example 8 can be rewritten as:

Example 9

Example 9 lists match locations that contain more than one Email addresses match and at least one IP Address match, but only if those match locations do not contain any Passport Number matches.

Grouping Operators

Grouping operators can be used to combine a number of statements into a single logical statement, or to alter the precedence of operations. Group statements by surrounding them with parentheses ( ).

Syntax

( )

Example 10

For Example 10, the filter displays match locations that do not contain both SWIFT Code and International Bank Account Number (IBAN) matches. Match locations that meet any of the following conditions will be displayed for this filter:

• Contains no SWIFT Code and no International Bank Account Number (IBAN).
• Contains SWIFT Code but no International Bank Account Number (IBAN).
• Contains International Bank Account Number (IBAN) but no SWIFT Code.

Example 11

In Example 11, scanned locations are checked if they contain:

• At least one Personal Names (English) and at least one Date of Birth match, or
• At least one License Number match.

Because the AND operator has a higher precedence than the OR operator, the AND operation in [Personal Names (English)] AND [Date Of Birth] is evaluated first.

The below expression is equivalent to Example 11. While Example 11 uses implicit operator precedence, this example uses it explicitly:

Example 12

Example 12 shows how the operator precedence from Example 11 can be modified with grouping operators. Match locations that meet any of the following conditions will be displayed for this filter:

• Contain at least one Date Of Birth and one License Number.
• Contain at least one Date Of Birth and one Personal Names (English).

Remediating Matches While Using Advanced Filters

When performing remediation on selected matches, Advanced Filters are ignored. To change the scope of remedial action, restrict the number of match locations selected with the location filters.

See Filter Targets and Locations and Remedial Action for more information.