KeyError: 'DW_AT_byte_size' error on Ubuntu-Linux 4.15.0_48-generic system using Python 2.7.12

[dmwharris]

Hello, after creating a volatility profile for an Ubuntu-Linux 4.15.0_48-generic system using version 2.6 and running it against a LiME sample created with

insmod lime-4.15.0-48-generic.ko "path=/home/developer/lime-4.15.0-48-generic.lime format=lime"

I get the following error:

root@nuc2:/home/developer/Downloads/volatility-master# python vol.py --profile=LinuxUbuntu4_15_0-48-genericx64 -f /home/developer/lime-4.15.0-48-generic.lime modules
Volatility Foundation Volatility Framework 2.6
Traceback (most recent call last):
File "vol.py", line 192, in
main()
File "vol.py", line 183, in main
command.execute()
File "/home/developer/Downloads/volatility-master/volatility/commands.py", line 116, in execute
if not self.is_valid_profile(profsself._config.PROFILE):
File "/home/developer/Downloads/volatility-master/volatility/plugins/overlays/linux/linux.py", line 216, in init
obj.Profile.init(self, *args, **kwargs)
File "/home/developer/Downloads/volatility-master/volatility/obj.py", line 862, in init
self.reset()
File "/home/developer/Downloads/volatility-master/volatility/plugins/overlays/linux/linux.py", line 227, in reset
self.load_vtypes()
File "/home/developer/Downloads/volatility-master/volatility/plugins/overlays/linux/linux.py", line 264, in load_vtypes
vtypesvar = dwarf.DWARFParser(dwarfdata).finalize()
File "/home/developer/Downloads/volatility-master/volatility/dwarf.py", line 71, in init
self.feed_line(line)
File "/home/developer/Downloads/volatility-master/volatility/dwarf.py", line 162, in feed_line
self.process_statement(**parsed) #pylint: disable-msg=W0142
File "/home/developer/Downloads/volatility-master/volatility/dwarf.py", line 204, in process_statement
self.vtypes[name] = [ int(data['DW_AT_byte_size'], self.base), {} ]
KeyError: 'DW_AT_byte_size'

Python version is 2.7.12

I was using the dwarfdump associated with Ubuntu 16.01 xenial, which as 20120410-2+deb7u2build0.16.04.1.

Is this because version 2.6 doesn't support Linux 4.15.0-48-generic? Or should I use a later version of dwarfdump?

[pagabuc]

Hey @dmwharris, can you please share the profile? And yes, I suggest to try with another version of dwarfdump!

[shehreyarahmedkohati]

Hi also getting some what same error,
I am using Volatility 2.6 to analyze a memory image from a Ubuntu 16.04 server captured using and in Lime format. I am getting the below error persistently. I have made a several profiles for the OS using the System map details.
Error:
sudo python vol.py -f "ram.lime" --profile=LinuxUbuntu1604-36x64 linux_pslist
Volatility Foundation Volatility Framework 2.6
*** Failed to import volatility.plugins.registry.shutdown (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.getservicesids (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.timeliner (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.malware.apihooks (NameError: name 'distorm3' is not defined)
*** Failed to import volatility.plugins.malware.servicediff (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.userassist (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.getsids (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.shellbags (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.evtlogs (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.tcaudit (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.shimcache (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.dumpregistry (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.lsadump (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.malware.threads (NameError: name 'distorm3' is not defined)
*** Failed to import volatility.plugins.mac.apihooks_kernel (ImportError: No module named distorm3)
*** Failed to import volatility.plugins.registry.amcache (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.mac.check_syscall_shadow (ImportError: No module named distorm3)
*** Failed to import volatility.plugins.malware.svcscan (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.auditpol (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.ssdt (NameError: name 'distorm3' is not defined)
*** Failed to import volatility.plugins.registry.registryapi (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.mac.apihooks (ImportError: No module named distorm3)
*** Failed to import volatility.plugins.envars (ImportError: No module named Crypto.Hash)
Traceback (most recent call last):
File "vol.py", line 192, in
main()
File "vol.py", line 183, in main
command.execute()
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/plugins/linux/common.py", line 64, in execute
commands.Command.execute(self, *args, **kwargs)
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/commands.py", line 116, in execute
if not self.is_valid_profile(profsself._config.PROFILE):
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/plugins/overlays/linux/linux.py", line 216, in init
obj.Profile.init(self, *args, **kwargs)
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/obj.py", line 862, in init
self.reset()
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/plugins/overlays/linux/linux.py", line 227, in reset
self.load_vtypes()
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/plugins/overlays/linux/linux.py", line 264, in load_vtypes
vtypesvar = dwarf.DWARFParser(dwarfdata).finalize()
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/dwarf.py", line 71, in init
self.feed_line(line)
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/dwarf.py", line 162, in feed_line
self.process_statement(**parsed) #pylint: disable-msg=W0142
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/dwarf.py", line 204, in process_statement
self.vtypes[name] = [ int(data['DW_AT_byte_size'], self.base), {} ]
KeyError: 'DW_AT_byte_size'

Any help would be appreciated!

[atcuno]

Can you please confirm that you are using the latest master checkout of Volatility from here (github)? Line 204 does not match your backtrace: https://github.com/volatilityfoundation/volatility/blob/master/volatility/dwarf.py#L204

[atcuno]

@shehreyarahmedkohati (and others), please git pull and try to run Volatility with your profiles again. I made an update to dwarf.py to hopefully address everyone at once:

7b3f52b

[atcuno]

Closed this by accident.

[atcuno]

@olifre please git pull again and re-try

[olifre]

@atcuno That's interesting! Now it runs for ~5 minutes and uses some GB of memory, so it seems the dwarf parsing went fine.
Then, I end up at a (Pdb) prompt. Typing w, I get:

Volatility Foundation Volatility Framework 2.6.1
> /somewhere/volatility/plugins/overlays/linux/linux.py(262)_merge_anonymous_members()
-> raise exceptions.VolatilityException("Inconsistent linux profile - unable to look up " + str(e))
(Pdb) w
  /somewhere/vol.py(192)<module>()
-> main()
  /somewhere/vol.py(183)main()
-> command.execute()
  /somewhere/volatility/plugins/linux/common.py(67)execute()
-> commands.Command.execute(self, *args, **kwargs)
  /somewhere/volatility/commands.py(116)execute()
-> if not self.is_valid_profile(profs[self._config.PROFILE]()):
  /somewhere/volatility/plugins/overlays/linux/linux.py(218)__init__()
-> obj.Profile.__init__(self, *args, **kwargs)
  /somewhere/volatility/obj.py(862)__init__()
-> self.reset()
  /somewhere/volatility/plugins/overlays/linux/linux.py(232)reset()
-> self.load_vtypes()
  /somewhere/volatility/plugins/overlays/linux/linux.py(270)load_vtypes()
-> self._merge_anonymous_members(vtypesvar)
> /somewhere/volatility/plugins/overlays/linux/linux.py(262)_merge_anonymous_members()
-> raise exceptions.VolatilityException("Inconsistent linux profile - unable to look up " + str(e))

[olifre]

@atcuno Do I interpret correctly that this (likely) means this issue is solved and we now have volatilityfoundation/volatility3#222 reproduced with Volatility 2?

[atcuno]

It is not solved. I am working on it, but getting those new types to parse is being difficult.

[olifre]

Thanks! No worries, I was just interpreting this as being back to the original issue now (for which we are still unsure whether the dump has some issues), but since that's not the case, for sure take your time implementing those new types.