wiki:mork_database

Lire les .msf (Thunderbird)

Les fichier msf (mail summary file) sont encodé en mork (https://en.wikipedia.org/wiki/Mork_(file_format)).

Usage dans Thunderbird

Dans Thunderbird, le but de ces fichiers est de présenter un index des courriels présents dans les boites mail.

  • Si ce fichier n'existe pas, il est créé en relisant l'ensemble du dossier courant,
  • s'il est corrompu ou que la création échoue, le message "Impossible d'ouvrir le fichier de résumé pour <DOSSIER> sur <COMPTE>. Il se peut qu'il y ait une erreur sur le disque ou que le chemin complet soit trop long." est affiché.
  • il peut ne pas être complètement à jour, dans ce cas l'affichage de la boite courriel ne correspond pas à aux fichiers réellement présents (exemple: des fichiers marqués supprimés sont encore visibles, ou le contraire, des fichiers sensés être là ne sont pas visibles).
  • Lorsqu'on fait un clic droit sur un dossier et qu'on clic sur propriétés puis sur réparer le dossier on force la réécriture de ce fichier d'index .msf

Détail d'implémentation (copie du code de Thunderbird sur github):

cf doc: https://thunderbird-interfaces.readthedocs.io/en/latest/nsIMsgDBService.html#methods

Lire le contenu d'un fichier .msf

D'abord installer le module perl Mozilla::Mork (cf: https://metacpan.org/release/KRIPT/Mozilla-Mork-0.01/view/lib/Mozilla/Mork.pm) 

perl -MCPAN -e shell
install Mozilla::Mork

Puis créer le fichier perl suivant read_msf.pl avec ce contenu (inspiré du site ci dessus): 

#! /usr/bin/env perl
 
use Mozilla::Mork;
use Data::Dumper;
 
$file = $ARGV[0];
unless ($file) { die "Usage: $0 <filename>\n"; }
#get a reference to an array of hash's
my $MorkDetails = Mozilla::Mork->new($file);
my $results = $MorkDetails->ReturnReferenceStructure();
 
print Dumper($results);

Donner les droits d'execution 

chmod +x read_msf.pl

Usage: 

./read_msf.pl <chemin_vers_fichier_msf>

Note: à la première installation, le shell CPAN installe des variables d'environnement dans ~/.bashrc pensez à recharger le fichier avant de lancer le script.

Exemple de sortie

Exemple un dossier ne contenant que 2 courriels: “ceci est un test” (lu) et “test non lu” (non lu):

Le .msf original: 

// <!-- <mdb:mork:z v="1.4"/> -->
< <(a=c)> // (f=iso-8859-1)
  (B8=imapFlags)(B9=highestRecordedUID)
  (BA=ns:msg:db:row:scope:pending:all)(BB=ns:msg:db:table:kind:pending)
  (BC=dateReceived)(BD=ProtoThreadFlags)
  (BE=dobayes.mailnews@mozilla.org#junk.empty)
  (BF=dobayes.mailnews@mozilla.org#junk)(C0=junkscore)(C1=keywords)
  (C2=imageSize)(C3=recipient_names)(C4=folderName)(C5=storeToken)
  (C6=junkscoreorigin)(C7=junkpercent)(C8=preview)(C9=pseudoHdr)
  (CA=ns:msg:db:row:scope:ops:all)(CB=ns:msg:db:table:kind:ops)(CC=op)
  (CD=msgKey)(CE=opFlags)(CF=newFlags)(D0=srcFolderURI)(D1=srcMsgKey)
  (80=ns:msg:db:row:scope:msgs:all)(81=subject)(82=sender)(83=message-id)
  (84=references)(85=recipients)(86=date)(87=size)(88=flags)(89=priority)
  (8A=label)(8B=numLines)(8C=ccList)(8D=bccList)(8E=msgThreadId)
  (8F=threadId)(90=threadFlags)(91=threadNewestMsgDate)(92=children)
  (93=unreadChildren)(94=threadSubject)(95=msgCharSet)
  (96=ns:msg:db:table:kind:msgs)(97=ns:msg:db:table:kind:thread)
  (98=ns:msg:db:table:kind:allthreads)
  (99=ns:msg:db:row:scope:threads:all)(9A=threadParent)(9B=threadRoot)
  (9C=msgOffset)(9D=offlineMsgSize)
  (9E=ns:msg:db:row:scope:dbfolderinfo:all)
  (9F=ns:msg:db:table:kind:dbfolderinfo)(A0=numMsgs)(A1=numNewMsgs)
  (A2=folderSize)(A3=expungedBytes)(A4=folderDate)(A5=highWaterKey)
  (A6=mailboxName)(A7=UIDValidity)(A8=totPendingMsgs)
  (A9=unreadPendingMsgs)(AA=expiredMark)(AB=version)(AC=forceReparse)
  (AD=fixedBadRefThreading)(AE=onlineName)(AF=MRUTime)(B0=MRMTime)
  (B1=sortType)(B2=sortOrder)(B3=viewFlags)(B4=viewType)(B5=sortColumns)
  (B6=columnStates)(B7=highestModSeq)>
<(80=1)(94=fffffffe)(91=647729c3)(81=0)>
[1:m(^9B=1)(^8F^94)(^91^91)(^90=0)(^92=1)(^93=1)]
<(A8=2)>[2:m(^9B=2)(^8F=2)(^91=0)(^90=0)(^92=0)(^93=0)]

<(97=80)(8D=John Doe <john.doe@cliss21.org>)(8E
    =John DOE <john.doe@cliss21.org>)(8F=ceci est un test)
  (90=cc738647-4cb6-6942-8fc3-308174672f77@cliss21.org)(92=82f)(93
    =ffffffff)(95=993|John DOE)(86=882)(96=2178)(98=3a)(99=plugin)
  (9A=13)(9B=nonjunk)>
{1:^80 {(k^96:c)(s=9)} 
  [1(^88=80)(^82^8D)(^85^8E)(^81^8F)(^83^90)(^BC^91)(^86^91)(^89=1)
    (^87^92)(^9A^93)(^8E^94)(^BD=0)(^C3^95)(^9C^86)(^C5^96)(^9D^86)
    (^8B=3a)(^C0=0)(^C6^99)(^C7=13)(^8A=0)(^C1^9B)]}
{FFFFFFFE:^80 {(k^97:c)(s=9)1:m } 1 }
{FFFFFFFD:^99 {(k^98:c)(s=9)} [FFFFFFFE(^94^8F)]}

<(82=INBOX.test_msf)(83=8082014)(B0=1685531155)(9D=1685531154)(AF=10cc)
  (87=12)(88=)(89
    ={"selectCol":{"visible":false,"ordinal":"1"},"threadCol":{"visible":true,\
"ordinal":"1"},"flaggedCol":{"visible":true,"ordinal":"3"},"attachmentCol":{"v\
isible":true,"ordinal":"5"},"subjectCol":{"visible":true,"ordinal":"7"},"unrea\
dButtonColHeader":{"visible":true,"ordinal":"9"},"senderCol":{"visible":false,\
"ordinal":"11"},"recipientCol":{"visible":false,"ordinal":"13"},"correspondent\
Col":{"visible":true,"ordinal":"15"},"junkStatusCol":{"visible":true,"ordinal"\
:"17"},"receivedCol":{"visible":false,"ordinal":"19"},"dateCol":{"visible":tru\
e,"ordinal":"21"},"statusCol":{"visible":false,"ordinal":"23"},"sizeCol":{"vis\
ible":false,"ordinal":"25"},"tagsCol":{"visible":false,"ordinal":"27"},"accoun\
tCol":{"visible":false,"ordinal":"29"},"priorityCol":{"visible":false,"ordinal\
":"31"},"unreadCol":{"visible":false,"ordinal":"33"},"totalCol":{"visible":fal\
se,"ordinal":"35"},"locationCol":{"visible":false,"ordinal":"37"},"idCol":{"vi\
sible":false,"ordinal":"39"},"deleteCol":{"visible":false,"ordinal":"13"}})
  (8A=2379)(8B=ee00)(8C=5ae7b0cb)>
{1:^9E {(k^9F:c)(s=9)} 
  [1(^AB=1)(^AC=0)(^AD=1)(^AE^82)(^88^83)(^A6^82)(^AF^B0)(^B0^9D)(^A0=1)
    (^A1=1)(^A5=2)(^A8=1)(^A9=1)(^A3^AF)(^B1=12)(^B2=1)(^B3=1)(^B4=0)
    (^B5=)(^B6^89)(^B7^8A)(^B8^8B)(^A7^8C)(^B9=1)]}

<(9F=24abad64-7e29-0766-d7f8-6c2cb444f905@cliss21.org)(9E=test non lu)
  (A0=64772a0b)(A6=38)(A7=...)(A5=84a)(A9=1104)(AA=4356)>
{1:^BA {(k^BB:c)(s=9)} 
  [1(^83^9F)(^82^8D)(^85^8E)(^81^9E)(^BC^A0)(^86^A0)(^C3^95)(^C0=0)
    (^C6^99)(^C7=38)(^C8^A7)(^9D^A5)(^9C^A9)(^88=80)(^C5^AA)(^89=1)]}

@$${B{@
< <(a=c)> // (f=iso-8859-1)
  (D2=customSortCol)(D3=gloda-id)(D4=gloda-dirty)(D5=notAPhishMessage)>
[1:m(^93=0)]
[-2:m(^9B=2)(^8F=2)(^91^A0)(^90=0)(^92=1)(^93=1)]

<(B3=81)(A1=7f7)>
{-1:^80 {(k^96:c)(s=9)} 
  [1(^88=81)]
  [-2(^88=80)(^82^8D)(^85^8E)(^81^9E)(^83^9F)(^BC^A0)(^86^A0)(^C3^95)
    (^C0=0)(^C6^99)(^C7=38)(^C8^A7)(^9D^A5)(^9C^A9)(^C5^AA)(^89=1)(^87^A1)
    (^9A^93)(^8E=2)(^BD=0)]}
{-2:^80 {(k^97:c)(s=9)2:m } 2 }[-2:^99(^94^9E)]
<(B2=$122)>[-1:^9E(^AB=1)(^AC=0)(^AD=1)(^AE^82)(^88^83)(^A6^82)(^AF^B0)
    (^B0^9D)(^A0=2)(^A1=1)(^A5=2)(^A8=0)(^A9=0)(^A3^AF)(^B1=12)(^B2=2)
    (^B3=1)(^B4=0)(^B5^B2)(^B6^89)(^B7=0)(^B8^8B)(^A7^8C)(^B9=2)]
{-1:^BA {(k^BB:c)(s=9)} }
[-1:^BA]
@$$}B}@

@$${C{@
@$$}C}@

@$${D{@
@$$}D}@

@$${E{@
<(B4=2621d)>[1:^80(^D3^B4)]
<(B5=26220)>[2:^80(^D3^B5)]
<(B6=1685531165)>[1:^9E(^AF^B6)]
@$$}E}@

@$${F{@
@$$}F}@

La version décodée: 

$VAR1 = [
          {
            'junkscore' => '0',
            'offlineMsgSize' => '84a',
            'threadId' => '2',
            'dateReceived' => '64772a0b',
            'date' => '64772a0b',
            'ProtoThreadFlags' => '0',
            'priority' => '1',
            'recipient_names' => '993|John DOE',
            'recipients' => 'John DOE <john.doe@cliss21.org>',
            'threadSubject' => 'test non lu',
            'threadRoot' => '2',
            'sender' => 'John Doe <john.doe@cliss21.org>',
            'junkscoreorigin' => 'plugin',
            'subject' => 'test non lu',
            'msgThreadId' => '2',
            'flags' => '80',
            'size' => '7f7',
            'gloda-id' => '26220',
            'msgOffset' => '1104',
            'storeToken' => '4356',
            'unreadChildren' => '1',
            'preview' => '...',
            'children' => '1',
            'threadParent' => 'ffffffff',
            'message-id' => '24abad64-7e29-0766-d7f8-6c2cb444f905@cliss21.org',
            'junkpercent' => '38',
            'threadFlags' => '0',
            'threadNewestMsgDate' => '64772a0b'
          },
          {
            'threadSubject' => 'ceci est un test'
          },
          {
            'flags' => '8082014',
            'numNewMsgs' => '1',
            'label' => '0',
            'subject' => 'test non lu',
            'junkscoreorigin' => 'plugin',
            'sortType' => '12',
            'numMsgs' => '2',
            'gloda-id' => '2621d',
            'size' => '82f',
            'fixedBadRefThreading' => '1',
            'priority' => '1',
            'recipient_names' => '993|John DOE',
            'expungedBytes' => '10cc',
            'ProtoThreadFlags' => '0',
            'dateReceived' => '64772a0b',
            'junkscore' => '0',
            'threadId' => 'fffffffe',
            'offlineMsgSize' => '84a',
            'forceReparse' => '0',
            'threadRoot' => '1',
            'sender' => 'John Doe <john.doe@cliss21.org>',
            'onlineName' => 'INBOX.test_msf',
            'sortOrder' => '2',
            'MRUTime' => '1685531165',
            'message-id' => '24abad64-7e29-0766-d7f8-6c2cb444f905@cliss21.org',
            'columnStates' => '{"selectCol":{"visible":false,"ordinal":"1"},"threadCol":{"visible":true,"ordinal":"1"},"flaggedCol":{"visible":true,"ordinal":"3"},"attachmentCol":{"visible":true,"ordinal":"5"},"subjectCol":{"visible":true,"ordinal":"7"},"unreadButtonColHeader":{"visible":true,"ordinal":"9"},"senderCol":{"visible":false,"ordinal":"11"},"recipientCol":{"visible":false,"ordinal":"13"},"correspondentCol":{"visible":true,"ordinal":"15"},"junkStatusCol":{"visible":true,"ordinal":"17"},"receivedCol":{"visible":false,"ordinal":"19"},"dateCol":{"visible":true,"ordinal":"21"},"statusCol":{"visible":false,"ordinal":"23"},"sizeCol":{"visible":false,"ordinal":"25"},"tagsCol":{"visible":false,"ordinal":"27"},"accountCol":{"visible":false,"ordinal":"29"},"priorityCol":{"visible":false,"ordinal":"31"},"unreadCol":{"visible":false,"ordinal":"33"},"totalCol":{"visible":false,"ordinal":"35"},"locationCol":{"visible":false,"ordinal":"37"},"idCol":{"visible":false,"ordinal":"39"},"deleteCol":{"visible":false,"ordinal":"13"}}',
            'keywords' => 'nonjunk',
            'threadFlags' => '0',
            'version' => '1',
            'viewType' => '0',
            'highestModSeq' => '0',
            'threadParent' => 'ffffffff',
            'mailboxName' => 'INBOX.test_msf',
            'msgThreadId' => 'fffffffe',
            'date' => '64772a0b',
            'numLines' => '3a',
            'highestRecordedUID' => '2',
            'imapFlags' => 'ee00',
            'recipients' => 'John DOE <john.doe@cliss21.org>',
            'unreadPendingMsgs' => '0',
            'totPendingMsgs' => '0',
            'UIDValidity' => '5ae7b0cb',
            'viewFlags' => '1',
            'MRMTime' => '1685531154',
            'sortColumns' => '2',
            'junkpercent' => '38',
            'threadNewestMsgDate' => '647729c3',
            'unreadChildren' => '0',
            'msgOffset' => '1104',
            'storeToken' => '4356',
            'highWaterKey' => '2',
            'children' => '1',
            'preview' => '...'
          }
        ];
  • wiki/mork_database.txt
  • Dernière modification : 31/05/2023 13:13
  • de vincent.adolphe