Planet Coaster modding... Is it possible?

Status
Thread Closed: Not open for further replies.

DrowningStar

D
I would really like to be able to put the Huntsman on another track with all of its car controls still working... Hopefully one day this becomes a reality! Really excited about the future of modding.
 

dragons

D
rrobnett said:
After spending several days staring at a hex editor I have made a few discoveries.

1) Do not use Cobra Overlay Extractor. It might give some insight as to where to start looking. Apart from that it has some major flaws.

2) I am 99% sure that I can display the structure of all OVL files in a hex editor. This is what has taken the most amount of time to establish.

3) The structure of the ovl files follows the same pattern, but what sections each ovl file contains can very widely. Drilling down into these differences will be my next step, along with writing a procedure to display this info in a viable format.

4) The OVL file does tell you when there will be an associated OVS file. Now to figure out why some files have them and others do not.

I know it does not seem like there is much here and while it may not allow us to get any UGC into the game. It may provide us the means by which to modify the behavior of existing content. It would also be cool to find out how the ovl files are wired into the game and is it possible to get a new ovl file to be accepted by the game.
Click to expand...

I spent a couple of days looking into the ovl and ovs files a while back and was able to load both of them but was sidetracked before spending too much time working on actually dumping the data files out of them. I can tell you that the ovl file header does tell you if it has an ovs file and how many 'archives' are actually in it. So far every file I've looked at has between 1 and 3 archive sections for lack of a better term. The first section is part of the .ovl file itself and the other two are part of the .ovs file. Below are some of the structures in c#. First is the OVLHeader, that is followed by the string table (uses StringTableSize), an array of 'loader' structures (file types, uses LoaderCount), then an array of the loader symbols (files, uses SymbolsToResolve). The archive names come after that (uses ArchiveNamesLength). After that is the archive header (uses ArchiveCount) then the 'dir' structures (uses DirCount). Four more structure arrays follow and use PartCount, OtherCount, UnknownCount, and ArchiveCount again. If you've done that stuff correctly then next part should be the compressed archive data. There's a ton of unknown entries in the structures listed below but hopefully it will help some :)

Code: 
// OVLHeader

public uint magicNumber;

        
        public byte VersionOrGame;/// Seems to always be 0x8
        public byte Unknown1a; // always 18 (12h) 
        public byte BigEndian; // 
        public byte Unknown1c; // always 1 // 
        public byte Unknown2a; // always 148 // 
        public byte Unknown2b; // always 32 // 
        public byte Unknown2c; // always 0 // 
        public byte Unknown2d; // always 0 // 
        public uint Unknown3; // always 0 // 
        public uint StringTableSize;
        public uint Unknown4; // always 0
        public uint OtherCount;
        public ushort DirCount;
        public ushort LoaderCount;
        public uint LoaderSymbolCount1;
        public uint LoaderSymbolCount2; /// Same as LoaderSymbolCount1??
        public uint PartCount;
        public uint ArchiveCount;
        public uint archiveHeaderTotal1; 
        public uint archiveHeaderTotal2; // ???
        public uint fsUnk1CountCombined; 
        public uint fsUnk2CountCombined; 
        public uint UnknownCount;
        public ushort Unknown16a; // always 0
        public ushort Unknown16b; // always 0
        public ushort Unknown17a; // always 0
        public ushort Unknown17b; // always 0
        public uint Unknown18; // always 0
        public uint ArchiveNamesLength; // Archive Names
        public uint FileCount3; /// Same as LoaderSymbolCount1 and LoaderSymbolCount 2??
        public uint TypeNamesLength;
        public uint Unknown22; // always 0
        public uint Unknown23; // always 0
        public uint Unknown24; // always 0
        public uint Unknown25; // always 0
        public uint Unknown26; // always 0
        public uint Unknown27; // always 0
        public uint Unknown28; // always 0
        public uint Unknown29; // always 0
        public uint Unknown30; // always 0
        public uint Unknown31; // always 0
        public uint Unknown32; // always 0
        public uint Unknown33; // always 0
        public uint Unknown34; // always 0

// OVLLoader

        public uint NameStringPointer;
        public uint Unknown1;
        public uint Hash;
        public uint LoaderType;
        public uint SymbolStart;
        public uint SymbolsToResolve;
 
// OVLLoaderSymbol

        public uint StringPointer;
        public uint Hash;
        public byte Type;
        public byte Unknown2;
        public byte LoaderIndex;
        public byte Unknown4;

// OVLArchiveHeader

        public uint nameIndex;
        public uint Block1a;
        public uint Block1b;
        public uint Block2a;
        public uint Block2b;
        public uint headerSubTypeCnt;
        public uint Block3b;
        public uint fsUnk1Count;
        public uint headerTypeCnt;
        public uint Block5a;
        public uint Block5b;
        public uint fsUnk2Count;
        public uint Block6b;
        public uint fsUnk4Count;
        public uint fsUnk3Count;
        public uint compressedDataStart;
        public uint Unknown1;
        public uint CompressedDataSize;
        public uint DecompressedDataSize;
        public uint Unknown2;
        public uint Unknown3;
        public uint Header2Size;
        public uint Unknown5;

// OVLDir
  
        uint nameIndex;

// OVLPart

        public uint hash;
        public uint nameIndex;
        public uint unknown08;
        public uint unknown0C;
        public uint unknown10;
        public uint offset;

// OVLOther

        public uint unknown00;
        public uint nameIndex;
        public uint unknown08;
        public uint offset;

// OVLUnknown

        public uint unknown00;
        public uint unknown04;
        public uint unknown08;
        public uint offset;

// OVLArchive2

        public uint unknown00;
        public uint dataSsize;
 

rrobnett

rrobnett
Well time for another progress update. I have written a small app that will will read in the header files of OVL files and store that info. The info is then displayed in a data grid so that multiple files can be evaluated at 1 time. The code and app is stored on Github at this link https://github.com/rerobnett/PC-OVL-Viewer.git. I have also found how to get to the compressed data and I am able to decompress it. So at this point I am analyzing the header files and starting to figure out where the assets are stored in the data. I have coded in C# before but this is my first time making a stand alone app. If you have visual studio you can download the project and run it that way. Or run the compiled .exe file.There will be more information available in the future.


Thank you for posting you discoveries. I had already worked through the structure, but will check it against my code to see if there is any new info.

I have a feeling I am reinventing the wheel here. I am not sure why some people are waiting for me to figure it all out on my own before they contribute their findings. This seems like a huge waste of energy. If there is information out there that could move this endeavor along please consider letting the rest of us in on it.. I don't need to get to the finish line first. I just want to see the finish line.

For others interested in contributing, Dragons' post is worth having a look at. It looks fairly complete. I will know more after I check it against my code.
 
Last edited:

dragons

D
I kind of stopped working on it once I heard rumors of UGC being added which was my main goal for figuring out the formats. I just realized I didn't include the compressed parts so here those are. It's even more incomplete than the OVL stuff but hopefully it will help some.

For the compressed parts, after it's decompressed, here's the structures I have:

OVSTypeHeader - read in archiveHeader.headerTypeCnt of them

Code: 
        public int hdrType = 0;
        public int subTypeCount = 0;
        public int offset = 0;

then each of the headers read in may have subheaders so you loop through each of the main headers and load header.subTypeCount of them. They are each 6 uint values. Assuming you treat these as 1 to 6, the 3rd one is the size of the data for that section and the 4th is the beginning offset. Some of the file structures that follow have section numbers and offsets in them and you have to use the 3 and 4 values to determine the real offsets.

Next up is the first of potentially 4 sections that together point to the actual data.

OVSFileDataHeader - there will be archiveHeader.fsUnk1Count of these

Code: 
        public uint fileNo;   
        public uint type;
        public uint size1;
        public uint unk4;
        public uint size2;
        public uint unk6;

The next section's layout depends on the info in the first section and I haven't had time to figure them all out but the total size will be archiveHeader.fsUnk2Count*8. Some of these are really 2 uint but others are 4 uint.

OVSFileSection3 - archiveHeader.fsUnk3Count of these

Code: 
        public uint fileNum = 0;
        public uint section = 0;
        public uint offset = 0;

OVSFileSection4 - archiveHeader.fsUnk4Count of these

Code: 
        public uint section1 = 0;
        public uint offset1 = 0;
        public uint section2 = 0;
        public uint offset2 = 0;

And there's some extra data after this that I don't know what it is for. Here's how to read it in:

Code: 
           unk1 = br.ReadUInt32();
            unk2 = br.ReadUInt32();

            long vvv = br.ReadInt64();
            Debug.Assert(vvv == 72624977462935551, "Error in OVS!");
            br.BaseStream.Position -= 16;

            long ttt1 = archiveHeader.Unknown1;

            extraHeader = new byte[ttt1];


            Array.Copy(pMem, br.BaseStream.Position, extraHeader, 0, ttt1);

            br.BaseStream.Position += ttt1;

And now you should be at the first section of actual data for the files :) Each loader type has it's own way of using the 4 sections together to load the data. I've figured out a few of them but haven't had time to look into it further. Hope all this helps.[/QUOTE]
 

gregorburns

G
rrobnett said:
I have a feeling I am reinventing the wheel here. I am not sure why some people are waiting for me to figure it all out on my own before they contribute their findings.
Click to expand...

I think we are all reinventing the wheel actually [tongue]

I've already explained why I'm reluctant to share my findings. That is not going to change anytime soon [tongue]
 

rrobnett

rrobnett
dragons said:
I kind of stopped working on it once I heard rumors of UGC being added which was my main goal for figuring out the formats. I just realized I didn't include the compressed parts so here those are. It's even more incomplete than the OVL stuff but hopefully it will help some.
Click to expand...

I hope the rumor is true. I would really like to work on UGC for this game. My reasons for exploring the OVL files are to try and modify the behavior of existing content. Rides, stalls and such.

Right now I can actually strip a file of ALL THE HEADER DATA, and I can successfully unpack the remaining data. I am also able to unpack the OVS files.

What is left now is to get at the data. I think I understand what type of data should be in each section. I just don't know how to interpret those sections. It might be staring me right in the face and I am overthinking it. I believe there are different types of OVL files. Model files, texture files, behavior files, settings files, audio files and even files that contain a mixture of these. Even though there is a compressed section for an audios OVL file, I think, the only thing that is there is info about the AUX file??

One of the things I am working on is trying to extract an image map from a file.

Well enough babbling for now.
 

Waverunner

W
dragons said:
I spent a couple of days looking into the ovl and ovs files a while back and was able to load both of them but was sidetracked before spending too much time working on actually dumping the data files out of them. I can tell you that the ovl file header does tell you if it has an ovs file and how many 'archives' are actually in it. So far every file I've looked at has between 1 and 3 archive sections for lack of a better term. The first section is part of the .ovl file itself and the other two are part of the .ovs file. Below are some of the structures in c#. First is the OVLHeader, that is followed by the string table (uses StringTableSize), an array of 'loader' structures (file types, uses LoaderCount), then an array of the loader symbols (files, uses SymbolsToResolve). The archive names come after that (uses ArchiveNamesLength). After that is the archive header (uses ArchiveCount) then the 'dir' structures (uses DirCount). Four more structure arrays follow and use PartCount, OtherCount, UnknownCount, and ArchiveCount again. If you've done that stuff correctly then next part should be the compressed archive data. There's a ton of unknown entries in the structures listed below but hopefully it will help some :)

Code: 
// OVLArchiveHeader

        public uint nameIndex;
        public uint Block1a;
        public uint Block1b;
        public uint Block2a;
        public uint Block2b;
        public uint headerSubTypeCnt;
        public uint Block3b;
        public uint fsUnk1Count;
        public uint headerTypeCnt;
        public uint Block5a;
        public uint Block5b;
        public uint fsUnk2Count;
        public uint Block6b;
        public uint fsUnk4Count;
        public uint fsUnk3Count;
        public uint compressedDataStart;
        public uint Unknown1;
        public uint CompressedDataSize;
        public uint DecompressedDataSize;
        public uint Unknown2;
        public uint Unknown3;
        public uint Header2Size;
        public uint Unknown5;
Click to expand...

Been playing around with the file structures for a while now off and on between uni time. I did a lot over the summer and it's good to see there's some other people working on it besides me! Your Archive format is off... by a lot. I'm not really sure how you managed to even get the archive to work for all the OVL's, but this is the correct size. I'm assuming that you got some additional helper fields in there that aren't being read in, but even then the types are a bit off. Either way, I believe it should be the following:

Code: 
uint name_offset

short unknown_04
short padding_06

uint unknown_08
uint unknown_0C
short unknown_10
short unknown_12

uint padding_14

uint unknown_18

uint file_count
uint symbol_count

uint compressed_data_start

uint unknown_28

uint compressed_data_size
uint uncompressed_data_size

uint padding_34

uint unknown_38

uint header2_size

uint unknown_40

So each archive has a constant size of 76 bytes. Incase anyone is wondering, I ran a simple c-progra across all the OVL's and if a variable returned always 0 then I just call it padding. If it breaks then I rename it to an unknown with it's position lol.

Some of those archive unknowns it looks like you found, I just haven't been able to validate it yet. I'm more intrigued by your work on the structure for the unzipped data as I spent weeks trying to sort it out. I was able to decipher out some of the LUA files using the ovl that contains the 18 something scripts, but beyond that I couldn't figure out a set format. I'm fairly new to figuring out file formats but it is quite fun when you figure out a value :). The rest of my structs are fairly accurate with yours in terms of the sizes, though I do have a couple different names for some of the variables if it helps.

These are the starting bytes I have before the strings_length. These are all static from what I could tell, but wasn't completley sure. I have it split out into it's own struct for mine.

Code: 
char id[4];
ubyte game;
ubyte version; // 12
ubyte BigEndian; // 1 if file uses big-endian, otherwise it uses little-endian

ubyte Unknown_1c; // Seen 1
ubyte Unknown_2a; // Seen 148
ubyte Unknown_2b; // Seen 32
ubyte Unknown_2c; // Seen 0
ubyte Unknown_2d; // Seen 0
uint Unknown_3;


Also in case it wasn't already for someone curious, the data is compressed using standard zlib (starting bytes 78 9C, 9C is compression type I think, haven't seen anything different than 9C for the game).

Thanks for your very detailed information on the decompressed data, it's late right now so I'm going to bed and will tackle some more tomorrow!
 

dragons

D
You're right, I used uint for all the fields but some are loaded in as uint16 values. Sorry, it's been a long time since I looked at it. I'll see about uploading my code somewhere but I'll have to rip out some non-open source code I've used first. Here's how I actually load it:

Code: 
            nameIndex = reader.ReadUInt32();
            name = stringTable.StringPointer(nameIndex);

            Block1a = reader.ReadUInt16();
            Block1b = reader.ReadUInt16();
            Block2a = reader.ReadUInt16();
            Block2b = reader.ReadUInt16();

            headerSubTypeCnt = reader.ReadUInt16();
            Block3b = reader.ReadUInt16();
            fsUnk1Count = reader.ReadUInt16();
            headerTypeCnt = reader.ReadUInt16();
            Block5a = reader.ReadUInt16();
            Block5b = reader.ReadUInt16();
            fsUnk2Count = reader.ReadUInt16();
            Block6b = reader.ReadUInt16();
            fsUnk4Count = reader.ReadUInt32();
            fsUnk3Count = reader.ReadUInt32();
            compressedDataStart = reader.ReadUInt32();
            Unknown1 = reader.ReadUInt32();
            CompressedDataSize = reader.ReadUInt32();
            DecompressedDataSize = reader.ReadUInt32();
            Unknown2 = reader.ReadUInt32();
            Unknown3 = reader.ReadUInt32();
            Header2Size = reader.ReadUInt32();
            Unknown5 = reader.ReadUInt32();

edit: I just saw that OVSTypeHeader is wrong as well, it's 2 uint16, not 3 uint32. The offset field I listed is generated in my code, it's not in the file. The others look ok though.
 
Last edited:

rrobnett

rrobnett
If you go to this link https://github.com/rerobnett/PC-OVL-Viewer.git

you will find that I have finished the functionality of the program. All the class files are done and the zlib decompression portion is done.

Basically what the app does is:

1) Reads the header, string tables, directory and file descriptors, the 2 archive descriptors, and a couple of other sections that I am not sure how to name,
2) The information is stored in a self contained data base and the header information is displayed in a data grid.
3) It will also decompress the data section of the file and output it to the same diractory as the input file. .OVE is appended to the decompressed file.
4) In order for the decompression to work you will need to get a copy of IONIC,ZLIB.DLL. and put it in the debug directory. Which is the directory the app is ran from.
5) The original files are not altered in anyway. But it is still best to work with copies of the game files.
6} the app will store the information for as many files as you want to read in.

I will put in more functionality as time permits, or if there is an interest in this by someone other than myself. You can download the program files and do with them as you wish

I am currently trying to understand the structure of the uncompressed files. I am making some headway, but the progress is slow.
 

rrobnett

rrobnett
gregorburns said:
You do know C# has a built in DeflateStream right? [wink]
Click to expand...

No I did not know that. I will look into it

The IONIC.ZLIB.DLL is now included with the app.

I will look into the built in stream to see how it works and what it is capable of, Thanks for the heads up .
 

gregorburns

G
rrobnett said:
No I did not know that. I will look into it

The IONIC.ZLIB.DLL is now included with the app.

I will look into the built in stream to see how it works and what it is capable of, Thanks for the heads up .
Click to expand...

It just works like a normal file stream, but compresses. To use zlib you can just skip the 2-byte header at the beginning since it uses the same algorithm.
 

rrobnett

rrobnett
I looked it over today. Very interesting. If I find a need I will use it the next time, The OVL viewer app works just fine for me with the IONIC.ZLIB. So I don't see much of a need to rewrite it at this point. If I update it at any point then I may consider redoing the decompression class.

Last night I did include the ability to decompress the OVS files.
 

dragons

D
It's been slow at work lately so I've had time to dig in a little more. So far I'm able to load txt, xml, databases, lua, and some textures. For the textures, the formats vary but they are usually DDS files with the header information removed. So far I've found BC1 and BC3 files, along with some non-compressed RGBA8 and RGBAF32 formats. There's a few formats I haven't had time to figure out like the normal maps. Some other notes, lua scripts appear to be compiled with a custom lua engine, the LUAC_FORMAT value is 2 instead of 0 and it's missing some size information.

The OVSFileSection4 section I posted earlier is actually a patch table. It is used to insert the correct offsets into the structure information pointed to by OVSFileSection3. The first section/offset pair is the location to patch and the second is the value to patch in. Similarly, in the OVL file, The OVLUnknown section actually links the textures in the first compressed section to the LOD versions in the second and third compressed sections. The first value is the offset and the second where to patch it in. The 3rd value appears to always be 0 so I suspect the second and third is really one 64bit entry.

Finally, the OVSFileSection3 usually points to a structure that has more information specific to the type of file. As an example, for lua files it has size information and the start and end offset of the script name. For others such as textures, it has an offset to yet another structure that contains dimensions and mipmap information. I must admit that section3 had me really confused until I figured out the patch table part [happy]

I've considered posting my code somewhere and would prefer to hear something official on it before doing so. I have some concerns, especially in the DLC area since the flags for that appear to be really obvious.
 

Robert K

R
I would like to know if it is possible to hack a park to remove the skirt? Because I don’t like that in the grassland scenario there is a slight cliff on the one end that can’t be removed. What about that remove border restrictions trick that was posted? Would that allow you to do anything with the skirt or access that area?
 

rrobnett

rrobnett
dragons said:
It's been slow at work lately so I've had time to dig in a little more. So far I'm able to load txt, xml, databases, lua, and some textures. For the textures, the formats vary but they are usually DDS files with the header information removed. So far I've found BC1 and BC3 files, along with some non-compressed RGBA8 and RGBAF32 formats. There's a few formats I haven't had time to figure out like the normal maps. Some other notes, lua scripts appear to be compiled with a custom lua engine, the LUAC_FORMAT value is 2 instead of 0 and it's missing some size information.

The OVSFileSection4 section I posted earlier is actually a patch table. It is used to insert the correct offsets into the structure information pointed to by OVSFileSection3. The first section/offset pair is the location to patch and the second is the value to patch in. Similarly, in the OVL file, The OVLUnknown section actually links the textures in the first compressed section to the LOD versions in the second and third compressed sections. The first value is the offset and the second where to patch it in. The 3rd value appears to always be 0 so I suspect the second and third is really one 64bit entry.

Finally, the OVSFileSection3 usually points to a structure that has more information specific to the type of file. As an example, for lua files it has size information and the start and end offset of the script name. For others such as textures, it has an offset to yet another structure that contains dimensions and mipmap information. I must admit that section3 had me really confused until I figured out the patch table part [happy]

I've considered posting my code somewhere and would prefer to hear something official on it before doing so. I have some concerns, especially in the DLC area since the flags for that appear to be really obvious.
Click to expand...

This is very useful information, I will be taking a serious look into this real soon. I must admit you are a lot further along than I am, Real life has been distracting me from any real time to work on this. I hope that will change in the next couple of weeks.
 

dragons

D
For those who have been looking into the formats, be aware that the 1.4 update changed the init.ovl files for each content pack. Now they are encrypted which takes care of the DLC for free issue but also probably makes it very difficult to add new objects to the game. Changing or tweaking existing objects should still be doable though.
 

AndyC1

A
dragons said:
For those who have been looking into the formats, be aware that the 1.4 update changed the init.ovl files for each content pack. Now they are encrypted which takes care of the DLC for free issue but also probably makes it very difficult to add new objects to the game. Changing or tweaking existing objects should still be doable though.
Click to expand...
Naah, nothing much should be included in the init overlays - certainly anything you need to edit to add new objects will live either in the main overlay itself (where the databases still live), or creating new overlays per object (which is still supported). I've been quiet on this as I need to check exactly what I'm allowed to say on the matter, but I promise the very select data we store in the newly encrypted init overlays is not important for modding.

Cheers

Andy
 
Status
Thread Closed: Not open for further replies.
Back
Top Bottom