xfs_bmbt_rec_64 leading to wrong blocks

Discussion:

Felipe Monteiro de Carvalho

2014-08-06 10:12:12 UTC

Hello,

I am writing a software which reads xfs partitions and it is working
well so far, but, at in a particular folder with 10k files, only 7k
files appear, the other ones don't.

I read an inode which has di_core.di_format = XFS_DINODE_FMT_BTREE

No problem here, the btree is trivial, it has only 1 element inside
it, which leads to a list of 82 extends.

Extends 0..52 aprox. are all great, and work fine.

Extent 53 (and other ones after that) point to very wierd memory areas
which don't match the pattern that I saw previously =(

Here is the hex data of the extents, the selected extent in nr 53:

Loading Image...

And here is debug information showing which values I extracted from this table:

i=0 FLocalExtent.StartOff=0 FLocalExtent.StartBlock=66748
FLocalExtent.BlockCount=1 lStartBlock=65212
i=1 FLocalExtent.StartOff=1 FLocalExtent.StartBlock=66758
FLocalExtent.BlockCount=2 lStartBlock=65222
i=2 FLocalExtent.StartOff=3 FLocalExtent.StartBlock=66772
FLocalExtent.BlockCount=1 lStartBlock=65236
....

i=51 FLocalExtent.StartOff=77 FLocalExtent.StartBlock=67468
FLocalExtent.BlockCount=1 lStartBlock=65932
i=52 FLocalExtent.StartOff=78 FLocalExtent.StartBlock=67479
FLocalExtent.BlockCount=1 lStartBlock=65943
i=53 FLocalExtent.StartOff=8388608 FLocalExtent.StartBlock=66749
FLocalExtent.BlockCount=1 lStartBlock=65213
i=54 FLocalExtent.StartOff=8388609 FLocalExtent.StartBlock=66783
FLocalExtent.BlockCount=2 lStartBlock=65247
...

Note how StartOff is suddenly so big! But I manually checked the bits
comparing to xfs_bmbt_rec_64 and the value written is that one. But
what sense does it make for StartOff to jump like that?

And why are the blocks pointing to a non-XD2D block?

Here is the block to which one of the valid extents point to:

Loading Image...

And here is a block to which one of the extents that I cannot
interpretate point to:

Loading Image...

and another one:
Loading Image...

Any ideas???

I already worked so much on this, but I cannot figure out =(

thanks,

--
Felipe Monteiro de Carvalho

Brian Foster

2014-08-06 11:34:48 UTC

Permalink

Post by Felipe Monteiro de Carvalho
Hello,
I am writing a software which reads xfs partitions and it is working
well so far, but, at in a particular folder with 10k files, only 7k
files appear, the other ones don't.
I read an inode which has di_core.di_format = XFS_DINODE_FMT_BTREE
No problem here, the btree is trivial, it has only 1 element inside
it, which leads to a list of 82 extends.
Extends 0..52 aprox. are all great, and work fine.
Extent 53 (and other ones after that) point to very wierd memory areas
which don't match the pattern that I saw previously =(
http://magnifier.sourceforge.net/temp/xfs/extent_with_xfs_bmbt_rec_64.png
i=0 FLocalExtent.StartOff=0 FLocalExtent.StartBlock=66748
FLocalExtent.BlockCount=1 lStartBlock=65212
i=1 FLocalExtent.StartOff=1 FLocalExtent.StartBlock=66758
FLocalExtent.BlockCount=2 lStartBlock=65222
i=2 FLocalExtent.StartOff=3 FLocalExtent.StartBlock=66772
FLocalExtent.BlockCount=1 lStartBlock=65236
....
i=51 FLocalExtent.StartOff=77 FLocalExtent.StartBlock=67468
FLocalExtent.BlockCount=1 lStartBlock=65932
i=52 FLocalExtent.StartOff=78 FLocalExtent.StartBlock=67479
FLocalExtent.BlockCount=1 lStartBlock=65943
i=53 FLocalExtent.StartOff=8388608 FLocalExtent.StartBlock=66749
FLocalExtent.BlockCount=1 lStartBlock=65213
i=54 FLocalExtent.StartOff=8388609 FLocalExtent.StartBlock=66783
FLocalExtent.BlockCount=2 lStartBlock=65247
...
Note how StartOff is suddenly so big! But I manually checked the bits
comparing to xfs_bmbt_rec_64 and the value written is that one. But
what sense does it make for StartOff to jump like that?

Assuming 4k blocks, 8388608 is 32G, which is the offset interval used
for some internal sections of directories on XFS. If this is a
directory, it could be a node/leaf or freelist block. Looking at
xfs_da_format.h, it looks like XFS_DIR2_LEAF_OFFSET points to this
offset. This block basically points to a btree of hashed entry names for
lookup. See here for basics:

http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure//tmp/en-US/html/Directories.html

... and here for a diagram:

http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure//tmp/en-US/html/Node_Directories.html

Post by Felipe Monteiro de Carvalho
And why are the blocks pointing to a non-XD2D block?
http://magnifier.sourceforge.net/temp/xfs/correct_block_xd2d.png
And here is a block to which one of the extents that I cannot
http://magnifier.sourceforge.net/temp/xfs/wierd_block_xd2d.png
http://magnifier.sourceforge.net/temp/xfs/wierd_xfs_dir2_data_entry.png

The 0xFEBE magic seems to confirm the second one as a node block.

Brian

Post by Felipe Monteiro de Carvalho
Any ideas???
I already worked so much on this, but I cannot figure out =(
thanks,
--
Felipe Monteiro de Carvalho
_______________________________________________
xfs mailing list
http://oss.sgi.com/mailman/listinfo/xfs

Felipe Monteiro de Carvalho

2014-08-06 12:31:11 UTC

Permalink

Thanks! Indeed that's it =)

One strange thing however is that in my xfs_dir2_data.h there is:

#define XFS_DIR2_DATA_ALIGN_LOG 3 /* i.e., 8 bytes */
#define XFS_DIR2_SPACE_SIZE (1ULL << (32 + XFS_DIR2_DATA_ALIGN_LOG))
#define XFS_DIR2_LEAF_SPACE 1
#define XFS_DIR2_LEAF_OFFSET (XFS_DIR2_LEAF_SPACE * XFS_DIR2_SPACE_SIZE)

That doesn't match the number that I am seeing here for the offset.
Here I see 1 << 23, but according to the headers it should be 1 << 35

--
Felipe Monteiro de Carvalho

Post by Brian Foster

Assuming 4k blocks, 8388608 is 32G, which is the offset interval used
for some internal sections of directories on XFS. If this is a
directory, it could be a node/leaf or freelist block. Looking at
xfs_da_format.h, it looks like XFS_DIR2_LEAF_OFFSET points to this
offset. This block basically points to a btree of hashed entry names for
http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure//tmp/en-US/html/Directories.html
http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure//tmp/en-US/html/Node_Directories.html

The 0xFEBE magic seems to confirm the second one as a node block.
Brian

--
Felipe Monteiro de Carvalho

Brian Foster

2014-08-06 14:23:30 UTC

Permalink

Post by Felipe Monteiro de Carvalho
Thanks! Indeed that's it =)
#define XFS_DIR2_DATA_ALIGN_LOG 3 /* i.e., 8 bytes */
#define XFS_DIR2_SPACE_SIZE (1ULL << (32 + XFS_DIR2_DATA_ALIGN_LOG))
#define XFS_DIR2_LEAF_SPACE 1
#define XFS_DIR2_LEAF_OFFSET (XFS_DIR2_LEAF_SPACE * XFS_DIR2_SPACE_SIZE)
That doesn't match the number that I am seeing here for the offset.
Here I see 1 << 23, but according to the headers it should be 1 << 35

We're talking about extent map data here right? If so, the offset on
disk is in filesystem block units. E.g., block 8388608 * 4096 == 32G ==
(1 << 35).

Brian

Post by Felipe Monteiro de Carvalho
--
Felipe Monteiro de Carvalho

Post by Brian Foster

Assuming 4k blocks, 8388608 is 32G, which is the offset interval used
for some internal sections of directories on XFS. If this is a
directory, it could be a node/leaf or freelist block. Looking at
xfs_da_format.h, it looks like XFS_DIR2_LEAF_OFFSET points to this
offset. This block basically points to a btree of hashed entry names for
http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure//tmp/en-US/html/Directories.html
http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure//tmp/en-US/html/Node_Directories.html

The 0xFEBE magic seems to confirm the second one as a node block.
Brian

--
Felipe Monteiro de Carvalho
_______________________________________________
xfs mailing list
http://oss.sgi.com/mailman/listinfo/xfs

Felipe Monteiro de Carvalho

2014-08-07 11:51:01 UTC

Permalink

Hello,

Thanks, that worked perfectly =)

My actual idea is to not only read the partition, but to also recover
deleted/corrupted files. With the information that you provided I
advanced a lot in this direction, and I can now recover the file
information in that memory area completely. I can also obtain the
inode nr (well 4 bytes of it are preserved, but its better then
nothing), but my current problem is with the inode itself. In a
deleted file the following changes are done by the linux driver in the
inode:

xfs_dinode_core_t.di_mode was changed to zero
xfs_dinode_core_t.di_format was changed from 1 (local) to 2 (extent)

But di_mode in particular is a key element as I am using it to
differentiate files from directories.

Is there anything else what I can use to differentiate files from
directories in case di_mode was corrupted? I am searching here but so
far I didn't find anything...

thanks,

--
Felipe Monteiro de Carvalho

Brian Foster

2014-08-07 12:56:23 UTC

Permalink

Post by Felipe Monteiro de Carvalho
Hello,
Thanks, that worked perfectly =)
My actual idea is to not only read the partition, but to also recover
deleted/corrupted files. With the information that you provided I
advanced a lot in this direction, and I can now recover the file
information in that memory area completely. I can also obtain the
inode nr (well 4 bytes of it are preserved, but its better then
nothing), but my current problem is with the inode itself. In a
deleted file the following changes are done by the linux driver in the
xfs_dinode_core_t.di_mode was changed to zero
xfs_dinode_core_t.di_format was changed from 1 (local) to 2 (extent)

These are set in xfs_ifree() when the inode is pulled from the unlinked
list and ultimately freed. Resetting these values certainly makes sense,
but I'm not aware of the reason why di_format is set as such.

Post by Felipe Monteiro de Carvalho
But di_mode in particular is a key element as I am using it to
differentiate files from directories.

In general you can't rely on on-disk data once the inode has been freed.
Perhaps you should start a new thread with some kind of write up about
what you're trying to accomplish and how you're going about it.

Brian

Post by Felipe Monteiro de Carvalho
Is there anything else what I can use to differentiate files from
directories in case di_mode was corrupted? I am searching here but so
far I didn't find anything...
thanks,
--
Felipe Monteiro de Carvalho
_______________________________________________
xfs mailing list
http://oss.sgi.com/mailman/listinfo/xfs