If you ask DFIR analysts, "What is best in life?", the answer you should hear is, "...creating timelines!" After all, industry luminaries such as Andrew said, "Time is the most important thing in life, and timelines are one of the most useful tools for investigation and analysis.", and Chris said, "The timeline is the central concept of all investigative work."
My previous blog post addressed USB-connected devices, but only from the perspective of Windows Event Logs. In this blog post, I wanted to include data from the Registry, incorporated in a timeline so that the various data sources could be viewed through a common lens, in a single pane of glass.
I stated by using wevtutil.exe to export current copies of the five Windows Event Logs to a central location. I then used reg.exe to do the same thing for the System hive. I then used my timeline process (outlined in several of my books) to create the events file from the six data sources; I used wevtx.bat to parse the Windows Event Logs, and three newly created RegRipper Pro plugins to parse the relevant data from the System hive. The specific keys, values and data parsed from the hive were based largely on Yogesh's blog post, and this academic paper posted at the ResearchGate site. I created the initial plugins, and then modified them to display TLN-format output, for inclusion in timelines.
For this research, there where three specific devices I was interested in...my iPod, my iPhone, and a SanDisk Cruzer USB thumb drive. After creating the overall events file, I used the "type" and "find" commands to look for events associated specifically with those devices, isolated each into their own individual "overlay" events file, and then created timelines from each of those events files. This approach makes it easy to "see" what's going on and create artifact constellations, as I don't have to filter out "noise" associated with other events, and I still have the overall events file that I refer to.
What I'm sharing below are partial timelines of events, just enough to demonstrate events based on intentionally limited data sources, so that initial artifact constellations can be developed. From this point, the constellations can be built out; for example, accessing files the SanDisk Cruzer will produce Windows shortcut files pointing to files on the "E:\" volume. Again, these timeline overlays are not complete, but are intended to demonstrate Registry artifacts associated with USB-connected devices alongside Windows Event Log artifacts.
iPodA while back, I inserted my iPod into my computer in order to retrieve music files, via iTunes, so that I could transfer them to my iPhone. I didn't think much about it at the time, but the connection was clearly "remembered" by Windows 10, specifically via the Registry.
Here are the events around the insertion:
Sun Jan 2 19:41:21 2022 Z REG - First Inserted - Apple iPod [6&3091e96e&0&0000] REG - First Install - Apple iPod [6&3091e96e&0&0000] EVTX Stewie - Microsoft-Windows-WPD-MTPClassDriver/1005;Apple Inc.,Apple iPod,4.3.5,40 REG - Last Inserted - Apple iPod [6&3091e96e&0&0000]
Sun Jan 2 19:41:15 2022 Z EVTX Stewie - Microsoft-Windows-DeviceSetupManager/112;iPod,{fc916355-34ea-555c-9e24-3c59f6125097},2,42,11
And here are the events around the removal of the device from the computer, a little more than 14 minutes later:
Sun Jan 2 19:55:46 2022 Z REG - Last Removal - Apple iPod [6&3091e96e&0&0000]
The completed message string for the "Microsoft-Windows-DeviceSetupManager/112" event above is:
Device 'Apple iPod' ({fc916355-34ea-555c-9e24-3c59f6125097}) has been serviced, processed 6 tasks, wrote 34 properties, active worktime was 11748 milliseconds.
I state this specifically because following the "Last Removal" event on 2 Jan 2022, the timeline contains an additional 9 events from 6 Jan to 22 May, all for the same "Microsoft-Windows-DeviceSetupManager/112" event records for the iPod, but the last three string entries are different. In every case, only 1 task is run, and the active worktime runs from 0 to 31 milliseconds. I know that the iPod was not plugged in during these times, and as such, this seems to be an artifact the installation process.
iPhoneI have connected my iPhone to this Windows 10 system via a USB cable, to transfer pictures from it, and to transfer music files to it, via iTunes. Here was see one such connection on 7 May 2022:
Sat May 7 14:16:35 2022 Z REG - Last Removal - @oem119.inf,iphone.appleusb.devicedesc%;Apple Mobile Device USB Composite Device [00008030000E6C6C11DA802E] REG - Last Removal - Apple iPhone [6&139bb8e1&1&0000]
Sat May 7 14:14:57 2022 Z EVTX Stewie - Microsoft-Windows-WPD-MTPClassDriver/1005;Apple Inc.,Apple iPhone,15.4.1,40 EVTX Stewie - Microsoft-Windows-DeviceSetupManager/112;Apple iPhone,{7e8068a1-2d62-53fb-8285-a12072dfa871},4,34,296
Sat May 7 14:14:56 2022 Z REG - Last Inserted - Apple iPhone [6&139bb8e1&1&0000] REG - Last Inserted - @oem119.inf,iphone.appleusb.devicedesc%;Apple Mobile Device USB Composite Device [00008030000E6C6C11DA802E]
There's information later in the timeline regarding another connection to the system, this time to copy pictures off of the iPhone. The "Last Inserted" and "Last Removal" events are from a different Registry key as seen above, as noted by the serial number in brackets at the end of the "event".
Fri Apr 15 16:23:13 2022 Z REG - Last Removal - @oem119.inf,iphone.appleusbmux.devicedesc%;Apple Mobile Device USB Device [6&139bb8e1&1&0001]
...
Fri Apr 15 16:19:02 2022 Z EVTX Stewie - Microsoft-Windows-WPD-MTPClassDriver/1005;Apple Inc.,Apple iPhone,15.4.1,40
Fri Apr 15 16:18:57 2022 Z EVTX Stewie - Microsoft-Windows-DeviceSetupManager/112;Apple iPhone,{7e8068a1-2d62-53fb-8285-a12072dfa871},4,34,140 REG - Last Inserted - @oem119.inf,iphone.appleusbmux.devicedesc%;Apple Mobile Device USB Device [6&139bb8e1&1&0001]
CruzerThe artifact constellation for the SanDisk Cruzer thumb drive is a bit different from that of the iPhone and the iPod. In this case, the events around the last time the device was inserted and then removed from the computer is less than a minute...
Mon May 16 22:07:08 2022 Z EVTX Stewie - Microsoft-Windows-Partition/1006;1,8208,262401,false,0,0,0,0,0,7,SanDisk,Cruzer,8.02,2443931D6C0226E3,... REG - Last Removal - SanDisk Cruzer USB Device REG - Last Removal - Cruzer [E:\]
Mon May 16 22:06:26 2022 Z EVTX Stewie - Microsoft-Windows-Ntfs/145;3,{1e09345e-d3d4-11e8-92fd-1c4d704c6039},2,E:,false,0,{fab772f6-83e6-5d5f-1086-740d39e45bff},8,SanDisk ,16,Cruzer ... EVTX Stewie - Microsoft-Windows-Partition/1006;1,8208,262401,false,0,0,0,512,8036285952,7,SanDisk,Cruzer,8.02,2443931D6C0226E3,Integrated : ...
Mon May 16 22:06:24 2022 Z EVTX Stewie - Microsoft-Windows-Partition/1006;1,8208,262401,false,0,0,0,0,0,7,SanDisk,Cruzer,8.02,2443931D6C0226E3,... EVTX Stewie - Microsoft-Windows-DeviceSetupManager/112;Cruzer,{81fa6fcf-bfc9-5887-bdbc-2cffb6be0b29},4,34,281 REG - Last Inserted - Cruzer [E:\] REG - Last Inserted - SanDisk Cruzer USB Device
Note that several of the events, particularly those from the Partition/Diagnostic Event Log, are shortened here for readability.
Registry Each of the above three devices appears in the Registry, specifically in the System hive, sometimes in multiple locations. For example, the SanDisk Cruzer thumb drive appears in both the USBStor and WPDBUSENUM subkeys.
From the USBStor key:Disk&Ven_SanDisk&Prod_Cruzer&Rev_8.02 2443931D6C0226E3&0 DeviceDesc : @disk.inf,%disk_devdesc%;Disk drive Mfg : @disk.inf,%genmanufacturer%;(Standard disk drives) Service : disk FriendlyName : SanDisk Cruzer USB Device First Install : 2021-09-09 17:37:15Z First Inserted : 2021-09-09 17:37:15Z Last Inserted : 2022-05-16 22:06:24Z Last Removal : 2022-05-16 22:07:08Z
From the WPDBUSENUM key:_??_USBSTOR#Disk&Ven_SanDisk&Prod_Cruzer&Rev_8.02#2443931D6C0226E3&0#{53f56307-b6bf-11d0-94f2-00a0c91efb8b} DeviceDesc : Cruzer FriendlyName : E:\ First Install : 2021-09-09 17:37:17Z First Inserted : 2021-09-09 17:37:17Z Last Inserted : 2022-05-16 22:06:24Z Last Removal : 2022-05-16 22:07:08Z
The Apple devices appear beneath the USB key, based on the vendor ID:VID_05AC&PID_129E b9e69c2c948d76fd3f959be89193f30a500a0d50 DeviceDesc : @oem119.inf,%iphone.appleusb.devicedesc%;Apple Mobile Device USB Composite Device Mfg : @oem119.inf,%aapl%;Apple, Inc. Service : usbccgp FriendlyName : @oem119.inf,%iPhone.AppleUSB.DeviceDesc%;Apple Mobile Device USB Composite Device First Install : 2022-01-02 19:41:16Z First Inserted : 2022-01-02 19:41:15Z Last Inserted : 2022-01-02 19:41:15Z Last Removal : 2022-01-02 19:55:46Z
VID_05AC&PID_129E&MI_00 6&3091e96e&0&0000 DeviceDesc : Apple iPod Mfg : Apple Inc. Service : WUDFWpdMtp FriendlyName : Apple iPod First Install : 2022-01-02 19:41:21Z First Inserted : 2022-01-02 19:41:21Z Last Inserted : 2022-01-02 19:41:21Z Last Removal : 2022-01-02 19:55:46Z
VID_05AC&PID_129E&MI_01 6&3091e96e&0&0001 DeviceDesc : @oem119.inf,%iphone.appleusbmux.devicedesc%;Apple Mobile Device USB Device Mfg : @oem119.inf,%aapl%;Apple, Inc. Service : WINUSB FriendlyName : @oem119.inf,%iPhone.AppleUsbMux.DeviceDesc%;Apple Mobile Device USB Device First Install : 2022-01-02 19:41:16Z First Inserted : 2022-01-02 19:41:16Z Last Inserted : 2022-01-02 19:41:16Z Last Removal : 2022-01-02 19:55:46Z
VID_05AC&PID_12A8 00008030000E6C6C11DA802E DeviceDesc : @oem119.inf,%iphone.appleusb.devicedesc%;Apple Mobile Device USB Composite Device Mfg : @oem119.inf,%aapl%;Apple, Inc. Service : usbccgp FriendlyName : @oem119.inf,%iPhone.AppleUSB.DeviceDesc%;Apple Mobile Device USB Composite Device First Install : 2022-01-02 19:56:40Z First Inserted : 2022-01-02 19:56:40Z Last Inserted : 2022-05-07 14:14:56Z Last Removal : 2022-05-07 14:16:35Z
VID_05AC&PID_12A8&MI_00 6&139bb8e1&1&0000 DeviceDesc : Apple iPhone Mfg : Apple Inc. Service : WUDFWpdMtp FriendlyName : Apple iPhone First Install : 2022-01-02 19:56:46Z First Inserted : 2022-01-02 19:56:46Z Last Inserted : 2022-05-07 14:14:56Z Last Removal : 2022-05-07 14:16:35Z
Additional ResourcesNote that per Yogesh's blog post, the "Microsoft-Windows-Kernel-PnP/Device Configuration" Event Log may also contain information about the connected devices.
One More ThingWhile I was doing some research for this blog post, I ran across this entry for event ID 112, albeit from the Microsoft-Window-TaskScheduler/Operational" Event Log. Once again, please stop referring to event records solely by their ID, and start including the event source, as well.
USB Devices Redux
Back in 2005, Cory Altheide and I published the first paper on tracking USB storage devices across Windows systems; at the time, the focus was Windows XP. A lot has happened since then...I know, that's an understatement...as the Windows platform has developed and expanded, initially with Vista, then Windows 7, and even with Windows 10 there have been developments that have come (and gone) just between the various Win10 builds.
With respect to USB devices in particular, not long ago, we (the community) became aware that the Microsoft-Windows-DriverFrameworks-UserMode/Operational Event Log contained quite a bit of information (see this post for event IDs to track) that a digital forensic analyst could use to determine if and when USB devices had been connected to (and disconnected from) the system. This was a pretty profound finding, and very valuable...and then, for some unknown reason, that Windows Event Log was disabled by default.
Also, in researching information for this topic, I found that the EMDMgmt key in the Software hive, which is associated with ReadyBoost and provided insight into USB-connected devices, is no longer available either. Okay, so one less artifact, one artifact removed from the constellation...now we just need to adapt.
This is really nothing new, to be honest. DFIR analysts need to be adaptable, regardless of whether we're in a consultant or FTE role. If you're a consultant, you're going to see a new environment on every engagement, and there will be new things to deal with and discover. A while back, a teammate discovered that the customer had LANDesk installed on their systems, and the software monitoring component recorded a great deal of information regarding executed processes right there in the Registry. It's not too different in an internal, FTE role, as you're likely going to run across legacy builds, software loads that haven't been/can't be updated for some reason, applications or packages users have installed specifically to meet the needs of their department or of a customer, etc. Further, we've seen various resources come and go; for example, we got used to having Registry hive backups available during investigations, and then we lost access to them; they're no longer available by default. In addition to the Microsoft-Windows-DriverFrameworks-UserMode/Operational Event Log, the Microsoft-Windows-TaskScheduler/Operational Event Log seems to be disabled by default. As such, when we find that a data source or artifact that we're familiar with and used to using is no longer available, we have to invest in determining an alternate means for determining the same or similar information; we have to rebuild those artifact constellations.
Something else that has developed over time alongside the development of the Windows platform is how USB devices are treated. For example, Nicole Ibrahim described some time ago how some USB connected devices, specifically smartphones, are treated differently based on the protocol used. Nicole's presentation on the topic can be found here.
The overall point is that we can no longer consider all USB-connected devices to be the same, and as such, we may need to look in different locations within the OS, including different locations within the Registry and within different Windows Event Logs, to find the information pursuant to our analysis goals. Pursuant to this, I sat down with my own local system and started going through the Windows Event Logs, via the Event Viewer, one at a time, looking for indications of connected devices. What I found was that records of connections were dispersed across multiple logs, depending upon the type of device connected (i.e., smartphone/digital camera, ext HDD w/ enclosure, thumb drive, etc.).
As a caveat, these event records are not exclusive; that is to say that the individual event source/ID pairs do not pertain solely to USB connected devices. In many cases, the same event source/ID pair was found to contain information specific to the local physical hard drive, as well as to the different volumes on that hard drive. Also, all of these events are for the latest build of Windows 10 only, because that's all I have to test against.
So here's a look at the five resources I found; keep in mind this is limited based on what I have available to test with, but it should serve as a good starting point...
Microsoft-Windows-WPD-MTPClassDriver/Operational.evtxEvent Source: WPD-MTPClassDriverEvent ID: 1005
Fig 1: Event ID 1005
Figure 1 illustrates where I'd connected my iPhone to the system to pull pictures; I also have entries for my iPod (yes, I still have an iPod...) where I wanted to transfer music to my iPhone. Due to the protocol used, this is also where we'd likely find digital cameras, as well.
Microsoft-Windows-DeviceSetupManager/AdminEvent Source: DeviceSetupManagerEvent ID: 112
Fig 2: Event ID 112
Figure 2 shows where I'd connected a SanDisk Cruzer thumb drive to the computer.
Microsoft-Windows-StorageSpaces-Driver/Operational.evtxEvent Source: StorageSpaces-DriverEvent ID: 207
Fig 3: Event ID 207
I shared figure 3 because this is specifically an external HDD, one of those "wallet" drives with a small form factor/enclosure. This device is small enough that it's powered from the USB connection; I don't have a larger enclosure that requires an additional power source to test against.
Microsoft-Windows-Partition/Diagnostic.evtxEvent Source: PartitionEvent ID: 1006
Fig 4: Event ID 1006, XML view
Figure 4 is a bit different, because the "friendly view" simply says, "for internal use only". However, taking a look at the XML view, we see that the SanDisk Cruzer thumb drive and it's serial number pops right out!
Microsoft-Windows-Ntfs/Operational.evtxEvent Source: NtfsEvent ID: 145, 142
Fig 5: Event ID 145
Figure 5 shows the Ntfs/145 event record (in part) from a previous connection of the SanDisk Cruzer thumb drive. Event ID 142 provides additional information, including regarding the volume assignment (C:, D:, F:, etc.), if the volume is bootable, etc., which can be used to tie to shellbags and Windows Portable Devices artifacts in the Registry.
RecommendationsFrom a forensic perspective, if you're interested in tracking USB devices connected to systems, I'd recommend enabling the Microsoft-Windows-DriverFrameworks-UserMode/Operational Event Log, forwarding those event records off of the system (for processing via a SIEM), as well as threat hunting or some other mechanism to ensure that if the log is disabled again that this is detected and responded to in the appropriate manner.
Note that enabling the Microsoft-Windows-DriverFrameworks-UserMode/Operational Event Log is as straightforward as setting the "Enabled" value in the HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WINEVT\Channels\Microsoft-Windows-DriverFrameworks-UserMode/Operational key to "1" and rebooting the system. As a tip to DFIR analysts who need to perform verification and to threat hunters, the reverse technique (setting the "Enabled" value to "0") can be used disable normally-available Windows Event Logs.
Additional Resources
HEFC Blog - Sunday Funday, Daily Blog #197USB Storage Device Forensics for Windows 10, circa 2017Computer Forensics CCIC Training, chapter 8, circa 2017
With respect to USB devices in particular, not long ago, we (the community) became aware that the Microsoft-Windows-DriverFrameworks-UserMode/Operational Event Log contained quite a bit of information (see this post for event IDs to track) that a digital forensic analyst could use to determine if and when USB devices had been connected to (and disconnected from) the system. This was a pretty profound finding, and very valuable...and then, for some unknown reason, that Windows Event Log was disabled by default.
Also, in researching information for this topic, I found that the EMDMgmt key in the Software hive, which is associated with ReadyBoost and provided insight into USB-connected devices, is no longer available either. Okay, so one less artifact, one artifact removed from the constellation...now we just need to adapt.
This is really nothing new, to be honest. DFIR analysts need to be adaptable, regardless of whether we're in a consultant or FTE role. If you're a consultant, you're going to see a new environment on every engagement, and there will be new things to deal with and discover. A while back, a teammate discovered that the customer had LANDesk installed on their systems, and the software monitoring component recorded a great deal of information regarding executed processes right there in the Registry. It's not too different in an internal, FTE role, as you're likely going to run across legacy builds, software loads that haven't been/can't be updated for some reason, applications or packages users have installed specifically to meet the needs of their department or of a customer, etc. Further, we've seen various resources come and go; for example, we got used to having Registry hive backups available during investigations, and then we lost access to them; they're no longer available by default. In addition to the Microsoft-Windows-DriverFrameworks-UserMode/Operational Event Log, the Microsoft-Windows-TaskScheduler/Operational Event Log seems to be disabled by default. As such, when we find that a data source or artifact that we're familiar with and used to using is no longer available, we have to invest in determining an alternate means for determining the same or similar information; we have to rebuild those artifact constellations.
Something else that has developed over time alongside the development of the Windows platform is how USB devices are treated. For example, Nicole Ibrahim described some time ago how some USB connected devices, specifically smartphones, are treated differently based on the protocol used. Nicole's presentation on the topic can be found here.
The overall point is that we can no longer consider all USB-connected devices to be the same, and as such, we may need to look in different locations within the OS, including different locations within the Registry and within different Windows Event Logs, to find the information pursuant to our analysis goals. Pursuant to this, I sat down with my own local system and started going through the Windows Event Logs, via the Event Viewer, one at a time, looking for indications of connected devices. What I found was that records of connections were dispersed across multiple logs, depending upon the type of device connected (i.e., smartphone/digital camera, ext HDD w/ enclosure, thumb drive, etc.).
As a caveat, these event records are not exclusive; that is to say that the individual event source/ID pairs do not pertain solely to USB connected devices. In many cases, the same event source/ID pair was found to contain information specific to the local physical hard drive, as well as to the different volumes on that hard drive. Also, all of these events are for the latest build of Windows 10 only, because that's all I have to test against.
So here's a look at the five resources I found; keep in mind this is limited based on what I have available to test with, but it should serve as a good starting point...
Microsoft-Windows-WPD-MTPClassDriver/Operational.evtxEvent Source: WPD-MTPClassDriverEvent ID: 1005
Fig 1: Event ID 1005Figure 1 illustrates where I'd connected my iPhone to the system to pull pictures; I also have entries for my iPod (yes, I still have an iPod...) where I wanted to transfer music to my iPhone. Due to the protocol used, this is also where we'd likely find digital cameras, as well.
Microsoft-Windows-DeviceSetupManager/AdminEvent Source: DeviceSetupManagerEvent ID: 112
Fig 2: Event ID 112Figure 2 shows where I'd connected a SanDisk Cruzer thumb drive to the computer.
Microsoft-Windows-StorageSpaces-Driver/Operational.evtxEvent Source: StorageSpaces-DriverEvent ID: 207
Fig 3: Event ID 207I shared figure 3 because this is specifically an external HDD, one of those "wallet" drives with a small form factor/enclosure. This device is small enough that it's powered from the USB connection; I don't have a larger enclosure that requires an additional power source to test against.
Microsoft-Windows-Partition/Diagnostic.evtxEvent Source: PartitionEvent ID: 1006
Fig 4: Event ID 1006, XML viewFigure 4 is a bit different, because the "friendly view" simply says, "for internal use only". However, taking a look at the XML view, we see that the SanDisk Cruzer thumb drive and it's serial number pops right out!
Microsoft-Windows-Ntfs/Operational.evtxEvent Source: NtfsEvent ID: 145, 142
Fig 5: Event ID 145Figure 5 shows the Ntfs/145 event record (in part) from a previous connection of the SanDisk Cruzer thumb drive. Event ID 142 provides additional information, including regarding the volume assignment (C:, D:, F:, etc.), if the volume is bootable, etc., which can be used to tie to shellbags and Windows Portable Devices artifacts in the Registry.
RecommendationsFrom a forensic perspective, if you're interested in tracking USB devices connected to systems, I'd recommend enabling the Microsoft-Windows-DriverFrameworks-UserMode/Operational Event Log, forwarding those event records off of the system (for processing via a SIEM), as well as threat hunting or some other mechanism to ensure that if the log is disabled again that this is detected and responded to in the appropriate manner.
Note that enabling the Microsoft-Windows-DriverFrameworks-UserMode/Operational Event Log is as straightforward as setting the "Enabled" value in the HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WINEVT\Channels\Microsoft-Windows-DriverFrameworks-UserMode/Operational key to "1" and rebooting the system. As a tip to DFIR analysts who need to perform verification and to threat hunters, the reverse technique (setting the "Enabled" value to "0") can be used disable normally-available Windows Event Logs.
Additional Resources
HEFC Blog - Sunday Funday, Daily Blog #197USB Storage Device Forensics for Windows 10, circa 2017Computer Forensics CCIC Training, chapter 8, circa 2017
Understanding Data Sources and File Formats
Following on the heels of my previous post regarding file formats and sharing the link to the post on LinkedIn, I had some additional thoughts that would benefit greatly from not blasting those thoughts out as comments to the original post, but instead editing and refining them via this medium.
My first thought was, is it necessary for every analyst to have deep, intimate knowledge of file formats? The answer to that is a resounding "no", because it's simply not possible, and not scalable. There are too many possible file formats for analysts to be familiar with; however, if a few knowledgeable analysts, ones who understand the value of the file format information to DFIR, CTI, etc., document the information and are available to act as resources, then that should suffice. With the format and it's value documented and reviewed/updated regularly, this should act as a powerful resource.
What is necessary is that analysts be familiar enough with data sources and file formats to understand when something is amiss, or when something is not presented by the tools, and know enough recognize that fact. From there, simple troubleshooting steps can allow the analyst to develop a thoughtful, reasoned question, and to seek guidance.
So, why is understanding file formats important for DFIR analysts?
1. Parsing
First, you need to understand how to effectively parse the data. Again, not every analyst needs to be an expert in all file formats - that's simply impossible. But, if you're working on Windows systems, understanding file formats such as the MFT and USN change journal, and how they can be tied together, is important. In fact, it can be critical to correctly answering analysis questions. Many analysts parse these two file separately, but David and Matt's TriForce tool allowed these files (and the $LogFile) to be automatically correlated.
So, do you need to be able to write an OLE parser when so many others already exist? No, not at all. However, we should have enough of an understanding to know that certain tools allow us to parse certain types of files, albeit only to a certain level. We should also have enough understanding of the file format to recognize that not all files that follow the format are necessarily going to have the same content. I know, that sounds somewhat rudimentary, but there are lot of new folks in the DFIR industry who don't have previous experience with older, perhaps less observed file formats.
Having an understanding of the format also allows us to ask better questions, particularly when it comes to troubleshooting an issue with the parser. Was something missed because the parser did not address or handle something, or is it because our "understanding" of the file format is actually a "misunderstanding"?
2. The "Other" Data
Second, understanding file formats provides insight into what other data the file format may contain, such as deleted data, "slack", and metadata. Several file formats emulate file systems; MS describes OLE files as "a file system within a file", and Registry hives are described as a "hierarchal database". Each of these file formats has their own method for addressing deleted content, as well as managing "slack". Further, many file formats maintain metadata that can be used for a variety of purposes. In 2002, an MSWord document contained metadata that came back to haunt Tony Blair's administration. More recently, Registry hive files were found to contain metadata that identified hives as "dirty", prompting further actions from DFIR analysts. Understanding what metadata may be available is also valuable when that metadata is not present, as observed recently in "weaponized" LNK files delivered by Emotet threat actors, in a change of TTPs.
3. Carving
Third, "file carving" has been an important topic since the early days of forensic analysis, and analysts have been asked to recover deleted files from a number of file systems. Recovering, or "carving" deleted files can be arduous and error prone, and if you understand file formats, it may be much more fruitful to carve for file records, rather than the entire file. For example, understanding the file and record structure of Windows 2000 and XP Event Logs (.evt files) allowed records to be recovered from memory and unallocated space, where carving for entire files would yield limited results, if any. In fact, understanding the record structure allowed for complete records to be extracted from "unallocated space" within the .evt files themselves. I used the successfully where an Event Log file header stated that there were 20 records in the file, but I was able to extract 22 complete records from the file.
Even today, the same holds true for other types of "records", including "records" such as Registry key and value nodes, etc. Rather than looking for file headers and then grabbing the subsequent X number of bytes, we can instead look for the smaller records. I've used this approach to extract deleted keys and values from the unallocated space within a Registry hive file, and the same technique can be used for other data sources, as well.
4. What's "In" The File
Finally, understanding the file format will help understand what should and should not be resident in the file. One example I like to look back on occurred during a PCI forensic investigation; an analyst on our team ran our process for searching for credit card numbers (CCNs) and stated in their draft report that CCNs were found "in" a Registry hive file. As this is not something we'd seen previously, this peaked our curiosity, and some of use wanted to take a closer look. It turned out that what had happened was this...the threat actor had compromised the system, and run their process for locating CCNs. At the time, the malware used would (a) dump process memory from the back office server process that managed CCN authorization and processing to a file, (b) parse the process memory dump with a 'compiled' Perl script that included 7 regex's to locate CCNs, and then (c) write the potential CCNs to a text file. The threat actor then compressed, encrypted, and exfiltrated the output file, deleting the original text file. This deleted text file then became part of unallocated space within the file system, and the sectors that comprised the file were available for reallocation.
Later, as the Registry hive file "grew" and new data was added, sectors from the file system were added to the logical structure of the file, and some of those sectors were from the deleted text file. So, while the CCNs were found "in" the logical file structure, they were not actually part of the Registry. The CCN search process we used at the time returned the "hit" as well as the offset within the file; a visual inspection of the file via a hex editor illustrated that the CCNs were not part of the Registry structure, as they were not found to be associated with any key or value nodes.
As such, what at first looked like a new threat actor TTP was really just how the file system worked, which had a significant impact on the message that was delivered to Visa, who "ran" the PCI Council at the time.
- 1
- 2
- 3
- …
- 31
- Next Page »