Serenity v3.30
Posted: Sun Apr 15, 2007 2:54 pm
I stumbled upon a mod that wouldn't work with Serenity, despite looking like it should...Turns out it created more than 16M of patch data for a single block...I guess I never really thought this would happen, so I added support for it...Looking at the map, it turned out the large patch block was caused by a poorly built map, rather than actually changing 16M worth of data...Had this patch been built differently, it would have cut around 26M (pre-compression) off the patch...Thought I'd explain somewhat how and why this happens...
Patches are done by a byte by byte comparison without any sync. So if you take the data abcdefgh, and change it to abcxefgh, the patch will store offset 4, change 1 byte to x...Now, if you take the same data, abcdefgh, and change it to abcdefghx, the patch will store offset 9, change 1 byte to x, or add 1 byte x to end of data, depending on the patch format...Now, because patches do not have sync, if you take the data abcdefgh, and change it to xabcdefgh, the patch is forced to store offset 1, change 9 bytes to xabcdefgh...Because the data is shifted, when creating the patch it will compare x to a, a to b, b to c, and so on, finding that every byte has changed...This is the same for all patch formats, ppf, sppf, or serenity...
Now, where these formats differ, is blocking...Blocking involves breaking data up into set blocks, which minimizes the effects of shifting to a single block...PPF is the base patch format, and because it is not file format specific, only has one block...What this means is, if a single byte is added to the beginning of the file, the shifting effect is seen throughout the entire file...SPPF steps this up a notch, by splitting halo 2 map files into 4 blocks...So any shifting done in any block, will be localized to that block...For example, if a single byte is added to the sound data, ppf would see the effects of shifting well into the meta block, making for extremely large patches...SPPF however, would only see shifting till shortly after the BSP data...So none of the bitmap data, or meta data would have to be included in the patch due to shifting...The major downfall with SPPF, is that the block sizes vary greatly, because their start/end points must be found within the map file header to maximize compatibility...
Now for Serenity...In compliant mode, Serenity will break the map into approximately 35 blocks, plus a separate block for every tag in the map...Thus any changes made to any tag, will be localized, and not effect other tags after it in the map...To have this kind of precision however, the map file must be nearly flawless, so that Serenity has access to all the proper data, and can find correct starting and ending points...Serenity in compliant mode will actually allow holes, as long as they are contained on either side with correct data...So a hole in the center of the bitmap data will be acceptable...Then there is non-compliant mode...This mode treats the entire meta as a single block...It is also much less strict on out of order data and holes, though the lowest offset for data must be in the correct order relative to the rest of the map...What this means, is that sound data can be injected anywhere in the map, as long as sound data exists at offset 2048 where the sound data should start...And model data can be anywhere in the map, as long as the first bit of model data exists after sound data, and before other data...It's fairly easy to get a map to work with non-compliant mode if attempted...
New in Serenity v3.30 is NFG Mode...This will basically use the same 4 points as an sppf patch for those of you who are poor excuses for modders, don't care about the size of mod you're forcing people to get, and in general horrible human beings...This will allow you to take full advantage of the features Serenity provides, though it will mean for larger patches than the other 2 Serenity modes, which will usually prevent from halomods hosting, and thus additions to the serenity repository...It also means SPPF can be phased out the same way PPF has been, and we can all sleep easier at night...
Patches are done by a byte by byte comparison without any sync. So if you take the data abcdefgh, and change it to abcxefgh, the patch will store offset 4, change 1 byte to x...Now, if you take the same data, abcdefgh, and change it to abcdefghx, the patch will store offset 9, change 1 byte to x, or add 1 byte x to end of data, depending on the patch format...Now, because patches do not have sync, if you take the data abcdefgh, and change it to xabcdefgh, the patch is forced to store offset 1, change 9 bytes to xabcdefgh...Because the data is shifted, when creating the patch it will compare x to a, a to b, b to c, and so on, finding that every byte has changed...This is the same for all patch formats, ppf, sppf, or serenity...
Now, where these formats differ, is blocking...Blocking involves breaking data up into set blocks, which minimizes the effects of shifting to a single block...PPF is the base patch format, and because it is not file format specific, only has one block...What this means is, if a single byte is added to the beginning of the file, the shifting effect is seen throughout the entire file...SPPF steps this up a notch, by splitting halo 2 map files into 4 blocks...So any shifting done in any block, will be localized to that block...For example, if a single byte is added to the sound data, ppf would see the effects of shifting well into the meta block, making for extremely large patches...SPPF however, would only see shifting till shortly after the BSP data...So none of the bitmap data, or meta data would have to be included in the patch due to shifting...The major downfall with SPPF, is that the block sizes vary greatly, because their start/end points must be found within the map file header to maximize compatibility...
Now for Serenity...In compliant mode, Serenity will break the map into approximately 35 blocks, plus a separate block for every tag in the map...Thus any changes made to any tag, will be localized, and not effect other tags after it in the map...To have this kind of precision however, the map file must be nearly flawless, so that Serenity has access to all the proper data, and can find correct starting and ending points...Serenity in compliant mode will actually allow holes, as long as they are contained on either side with correct data...So a hole in the center of the bitmap data will be acceptable...Then there is non-compliant mode...This mode treats the entire meta as a single block...It is also much less strict on out of order data and holes, though the lowest offset for data must be in the correct order relative to the rest of the map...What this means, is that sound data can be injected anywhere in the map, as long as sound data exists at offset 2048 where the sound data should start...And model data can be anywhere in the map, as long as the first bit of model data exists after sound data, and before other data...It's fairly easy to get a map to work with non-compliant mode if attempted...
New in Serenity v3.30 is NFG Mode...This will basically use the same 4 points as an sppf patch for those of you who are poor excuses for modders, don't care about the size of mod you're forcing people to get, and in general horrible human beings...This will allow you to take full advantage of the features Serenity provides, though it will mean for larger patches than the other 2 Serenity modes, which will usually prevent from halomods hosting, and thus additions to the serenity repository...It also means SPPF can be phased out the same way PPF has been, and we can all sleep easier at night...
