MySQL Data Recovery with TwinDB Undrop for InnoDB

MySQL Data Recovery with
TwinDB Undrop for InnoDB
Aleksandr Kuzminsky
TwinDB
Agenda
●
●
●
●
InnoDB files and their formats
Recovery corrupted databases
Recovery after DROP TABLE
Recovery compressed InnoDB tables
2
InnoDB files and their formats
ibdata1
●
InnoDB dictionary:
○
○
●
Undo
○
●
●
Table name -> index_id
Table structure (duplicates .frm)
Old versions of modified fields
Double write buffer
Pre-innodb_file_per_table tables
InnoDB Index
InnoDB page format
Size, bytes
FIL_HEADER
38
PAGE_HEADER
56
INFINUM+SUPREMUM RECORDS
varies
User records
varies
Free space
Page directory
FIL_TRAILER
varies
fixed
FIL_HEADER
Field
Size Offset
Comment
FIL_PAGE_SPACE
4
0 Space ID the page is in or checksum
FIL_PAGE_OFFSET
4
4 ordinal page number from start of space
FIL_PAGE_PREV
4
8 offset of previous page in key order
FIL_PAGE_NEXT
4
0C offset of next page in key order
FIL_PAGE_LSN
8
10 log serial number of page's latest log record
2
18 current defined types are: FIL_PAGE_INDEX,
FIL_PAGE_UNDO_LOG, FIL_PAGE_INODE,
FIL_PAGE_IBUF_FREE_LIST
8
1A "the file has been flushed to disk at least up to
this lsn" (log serial number), valid only on the
first page of the file
4
22 /* starting from 4.1.x this contains the
space id of the page */
FIL_PAGE_FILE_FLUSH_LSN
PAGE_HEADER (Part I)
Field
Size Offset Comment
PAGE_N_DIR_SLOTS
2
26 number of directory slots in the Page
Directory part; initial value = 2
PAGE_HEAP_TOP
2
28 record pointer to first record in heap
PAGE_N_HEAP
2
2A number of heap records; initial value = 2
PAGE_FREE
2
2C record pointer to first free record
PAGE_GARBAGE
2
2E "number of bytes in deleted records"
PAGE_LAST_INSERT
2
30 record pointer to the last inserted record
PAGE_DIRECTION
2
32 either PAGE_LEFT, PAGE_RIGHT, or
PAGE_NO_DIRECTION
PAGE_HEADER (Part II)
Field
PAGE_N_DIRECTION
PAGE_MAX_TRX_ID
Size
Offset
Comment
2
34 number of consecutive inserts in the same direction,
e.g. "last 5 were all to the left"
2
36 number of user records
8
38 the highest ID of a transaction which might have
changed a record on the page (only set for secondary
indexes)
2
40 level within the index (0 for a leaf page)
8
42 identifier of the index the page belongs to
PAGE_BTR_SEG_LEAF
10
4A "file segment header for the leaf pages in a B-tree"
(this is irrelevant here)
PAGE_BTR_SEG_TOP
10
54 "file segment header for the non-leaf pages in a Btree" (this is irrelevant here)
Record format (REDUNDANT)
Record format (COMPACT)
●
Recovery corrupted databases
InnoDB dictionary
●
SYS_TABLES
○
●
SYS_INDEXES
○
●
table_id => index_id
SYS_COLUMNS
○
●
human-readable name => table_id
table_id => table fields
SYS_FIELDS
○
index_id => index fields
Recover table structure (mysqlfrm)
find . -name '*.frm' | \
xargs mysqlfrm --basedir /usr/ --port 3000 | \
grep -v ^# | \
sed 's/^$/;/' | \
sed -e 's/\\(^) ENGINE=.*\\)/\\1;/' -e 's/^;$//' | \
sed -e 's/CREATE \\(.*VIEW.*\\)/CREATE \1;/'";
Recover table structure (sys_parser)
./sys_parser -u root -d test sakila/actor
CREATE TABLE `actor`(
`actor_id` SMALLINT UNSIGNED NOT NULL,
`first_name` VARCHAR(45) CHARACTER SET 'utf8' COLLATE 'utf8_general_ci' NOT NULL,
`last_name` VARCHAR(45) CHARACTER SET 'utf8' COLLATE 'utf8_general_ci' NOT NULL,
`last_update` TIMESTAMP NOT NULL,
PRIMARY KEY (`actor_id`)
) ENGINE=InnoDB;
stream_parser
Find InnoDB pages in stream of bytes:
./stream_parser -f /var/lib/mysql/ibdata1
./stream_parser -f /var/lib/mysql/sakila/actor.ibd
./stream_parser -f /dev/sda
c_parser
Fetch records from InnoDB page
./c_parser -6
-f pages-actor.ibd/FIL_PAGE_INDEX/0000000000000015.page \
-t sakila/actor.sql
●
Recovery after DROP TABLE
Recovery after DROP
Same as corrupted except:
1.
Records in the dictionary are deleted. Use -D with c_parser
2.
.ibd files will be deleted
3.
.frm files will be deleted
Undelete dictionary records
./c_parser -4Df pages-ibdata1/FIL_PAGE_INDEX/0000000000000001.
page \
-t dictionary/SYS_TABLES.sql \
> dumps/default/SYS_TABLES \
2> dumps/default/SYS_TABLES.sql
DROP and innodb_file_per_table
●
Keep MySQL files on a separate partition!
○
○
●
Easy to remount it read-only
No need to take image, ready to use
stream_parser -f /dev/sdb
Recovery structure
.frm file is gone, check sys_parser slide.
●
Recovery compressed InnoDB tables
Recovery compressed tables
●
●
Decompress each page
Recover as normal page
https://code.launchpad.net/~akuzminsky/percona-data-recovery-tool-for-innodb/decompress
https://recovery.twindb.com
Section Information