2.2 How File Reporter Works

File Reporter was developed to examine, report and analyze Windows file systems and the Microsoft 365 cloud and its potential petabytes of data—in other words, millions of files, folders and shares, scattered among the various storage devices and Microsoft 365 applications that make up your network. This reporting includes file content and the associated rights of these files, folders, and network shares.

To examine, report, and analyze this data efficiently, File Reporter disperses the work among a Web application, Engine, Agents, a Scan Processor, the RabbitMQ messaging broker, either a PostgreSQL or Microsoft SQL Server database, Microsoft Active Directory, and Microsoft Azure Active Directory.

Figure 2-1 File Reporter Work Process

2.2.1 Core Components

The following are core components of Micro Focus File Reporter.

Web Application

The Web application runs on top of Microsoft Internet Information Services (IIS) and is the means of all administrative interaction. Among other things, the Web application is responsible for:

  • Management of scan policies and report definitions

  • Generating Preview reports

  • Access to stored reports

  • All other management functions

Engine

The Engine is the mechanism that runs File Reporter and runs from a Windows Server host. The Engine does the following:

  • Schedules the scans that the Agents conduct

  • Compiles scans for inclusion in a report

  • Runs scheduled reports

  • Manages scan delegations to Agents

  • Sends notifications that File Reporter has completed a scan or generated a report

Database

The database stores information needed for generating reports. This information includes:

  • Cached Active Directory objects

  • Scans

  • Identity system information such as names of Active Directory domains and forests

  • Schedule information pertaining to scans and reports

  • Notification information

  • Report definitions

  • Scan history

  • Scan policies

  • Free space on shares

2.2.2 File System Scanning

The following are components associated with file system scanning.

Scan Processor

The Scan Processor does the following:

  • Processes file system scan files

  • Updates file system scan information in the database

Agents

Agents are compact programs that run on Microsoft Windows Server hosts. Agents can examine and report on NTFS file systems and OneDrive for Business, SharePoint Online, and Teams files stored in the Microsoft 365 cloud. Additionally, Agents examine and report on security, including file and folder permissions. For more information, see Section D.0, AgentFS Scan Capabilities.

IMPORTANT:For optimal results, you should install an Agent on every server that has a share you want to report on.

Agents cannot be installed on NAS devices or clustered storage. For File Reporter to report on these type of devices, Agents can be set up as proxy agents.

For performing file system scans (rather than file content scans), File Reporter provides AgentFS.

Scans

Through AgentFS, File Reporter scans a storage resource. A storage resource can be a Microsoft network share or a Network Attached Storage (NAS) device.

File system scans are indexed data that are specific to a storage resource. They are the means of generating a storage report or the means of analyzing data using the analytics tools. File system scans include comprehensive information on the file types users are storing, when files were created, when they were last modified, permission data on the folders where these files reside, and much more.

File Reporter collects file system scans from the Agents and sends them to the Engine. The Engine then sends the scans to the Scan Processor, which stores the scans in the database.

You can conduct scans at any time, but we recommend using a scheduled time after normal business hours to minimize the effect on network performance.

NOTE:Procedures for performing scans are documented in Section 5.0, Scheduling and Performing File System Scans.

2.2.3 File Content Scanning

The following are components associated with file content scanning.

ManagerFC

The ManagerFC service is responsible for the execution and management of file scan jobs. The service performs the following tasks when processing a scan job:

  • Enumeration of files in target paths

  • Submission of files to scan queues in the message broker based on filter criteria

  • Processing of scan results and update of result data to the database and scan result files

AgentFC

AgentFC performs file content scans. AgentFC is hosted on a Windows Server and performs content scans on files stored on Windows servers and NAS devices.

Scans

Through AgentFC, the RabbitMQ messaging broker, and ManagerFC, File Reporter performs, classifies, and categorizes file content scans. For example, content scans can identify files containing specified patters such as U.S. Social Security or credit card numbers.

2.2.4 Microsoft 365 Cloud Scanning

With the release of version 4.0, File Reporter extends the ability to report what files are being stored on your enterprise storage devices and who has access to these files, to reporting on the files and associated permissions located in Microsoft 365 cloud repositories for OneDrive for Business, SharePoint Online document libraries, and Teams document libraries.

Unlike scanning the network file system separately for File System, Permissions, and Volume Free Space, scans for files and associated permissions stored in the Microsoft 365 cloud are conducted simultaneously.

Reporting on Microsoft 365 is done through Custom Queries and report layouts available at filequerycookbook.com. The process is as easy as searching through the cookbook to see what type of report you want to generate, downloading it, pasting the query into the File Reporter Report Designer Query Editor, defining any needed paths or making desired modifications, and then laying out the report.

2.2.5 Reporting

When File Reporter has a scan, you can utilize it to generate a report. You can generate reports through the following means:

  • Built-in Reports

  • Custom Queries

Built-in Reports

Generating a built-in report is as simple as selecting the report type from a menu.

To generate a report, the Engine takes all of the needed scans that are applicable to the specifications of the report and consolidates them into a single report by indexing the applicable scans.

Table 2-1 Built-in Report Types

File System Reports

Security Reports

Trending Reports

Folder Summary

Assigned NTFS Permissions

Volume Free Space

Detail Reports

Permissions by Path

 

File Extension

Permissions by Identity

 

Duplicate Files

Historic NTFS Permissions

 

Date-Age

 

 

Owner

 

 

Storage Cost

 

 

Comparison

 

 

Directory Quota

 

 

Historic File System Comparison

 

 

File Reporter lets you present built-in reports in various formats including PDF, Microsoft Excel, RTF, HTML, TXT, and CSV. The product also includes built-in graphs for certain report types.

Figure 2-2 Sample Report in Graphical Format

Custom Query Reports

These reports allow administrators who are familiar with querying the database to generate very specific report data that might not be available through one of the built-in report types.

Custom Query report data can be further customized for layout and presentation from a Windows workstation with the Report Designer.

File content and Microsoft 365 reports are delivered as Custom Query reports.

Figure 2-3 Page from a Custom Query Report Designed with the Report Designer.

2.2.6 Client Tools

File Reporter provides the following Client Tools, designed to be run from a Windows workstation.

Data Analytics

In addition to extensive reporting options, File Reporter provides the ability to graphically analyze file system data using a variety of analytics tools that are available to administrators through the Client Tools.

Dashboard

The Dashboard lets you graphically analyze data from file system scans according to the filters that you specify.

Figure 2-4 Dashboard

Tree Map

The Tree Map lets you view graphical representations of hierarchical file system data and in the process, gain insight very quickly.

Figure 2-5 Tree Map

Pivot Grid

The Pivot Grid gives you the ability to visually analyze data according to combinations of variables.

Figure 2-6 Pivot Grid

Report Viewer

The Report Viewer lets you to view all stored reports locally from a Windows workstation. Because the Report Viewer utilizes the resources of the Windows workstation, rather than those of the Engine, the Report Viewer can display stored reports much faster in most instances.

Figure 2-7 Report Viewer