|
|
Post by fufu508 on Feb 9, 2019 23:29:34 GMT -5
Ok so I have a mostly working search. insert code hereimport io
import struct
import sims4.resources
import sims4.commands
def get_clip_name(resource: io.BytesIO) -> str:
resource.seek(0x38, io.SEEK_SET)
name_length: int = struct.unpack('i', resource.read(4))[0]
return resource.read(name_length).decode('ascii')
@sims4.commands.Command('clips', command_type=sims4.commands.CommandType.Live)
def clips(keyword:str="fufu508", _connection=None):
output = sims4.commands.CheatOutput(_connection)
clip_count = 0
clip_limit = 20
badkeys = 0
output('Gathering list of clips...')
key = 0
keys = sims4.resources.list(type=sims4.resources.Types.CLIP_HEADER)
output(f'{len(keys)} clips found.')
output(f'Searching for {keyword} in clip names...')
while (clip_count < clip_limit) and (key < len(keys)):
try:
resource_loader = sims4.resources.ResourceLoader(keys[key], sims4.resources.Types.CLIP_HEADER)
resource: io.BytesIO = resource_loader.load()
clip_name: str = get_clip_name(resource)
if keyword in clip_name:
output(f'key {key}: {clip_name}')
clip_count = clip_count + 1
except:
badkeys = badkeys + 1
key = key + 1
output(f'Limit reached. {clip_limit} clip(s) listed. Last key read: {key}')
output(f'Number of clips with keys that failed to read: {badkeys}')
But after finding the number of clips I specified, the game eventually closes (crashes).  |
|
|
|
Post by andrew on Feb 10, 2019 0:45:57 GMT -5
fufu508 In your post about writing to a file, the example you posted looks like it just lacks from io import openAs far as the crashing, it could be because my lazy example is not closing the resource stream. You could try auto-closing it with a "with" statement. ...
resource_loader = sims4.resources.ResourceLoader(keys[key], sims4.resources.Types.CLIP_HEADER)
with resource_loader.load() as resource:
clip_name: str = get_clip_name(resource)
...
As far as skipping non-cc clips, that could be tricky. The quick ways are like you tried (filter by creator name), filter by ':' or 'PosePack' if you want to show all clips created by pose packs. If you only need clips that were created as a pose pack, you could also read through the pose pack snippet tunings to find them all like the pose player does and skip the steps of reading from the clip headers. The get_clip_name function is just one I quickly wrote to simply get the name with as few lines as possible. You could get more information about the clip from there (as much as you see in the warehouse of Studio) with more complete parsing code, but things like the location of the package or whether it is CC are not available there. Also the sims4.resources.Type can be found in the decompiled scripts (sims4/resources).
|
|
|
|
Post by fufu508 on Feb 10, 2019 1:19:31 GMT -5
Thanks Andrew for the quick response! I'll dig into the details after I get some sleep... :sunny
I think the name search will be adequate for now.
|
|
|
|
Post by fufu508 on Feb 10, 2019 10:31:35 GMT -5
My ts4script still doesn't generate a file after explicitly importing open. I'm currently searching the EA decompiled content for evidence of how EA may be writing files.
I confirmed that open is built in. No need to import, at least not in the pyCharm environment.
scratch.py
f = open("scratchy.txt", "w+")
for a in range(1, 10):
f.write(f'line {a}\n')
f.close() Output of scratchy.txt
line 1
line 2
line 3
line 4
line 5
line 6
line 7
line 8
line 9
|
|
|
|
Post by fufu508 on Feb 10, 2019 11:19:45 GMT -5
 After messing with my scratch.py in the previous post, I realized I made a very noob mistake!
I used Windows path separator (\) instead of the forward slash (/). This worked as expected in game. Now to grab the animations... import sims4.commands
@sims4.commands.Command('proto', command_type=sims4.commands.CommandType.Live)
def proto(_connection=None):
output = sims4.commands.CheatOutput(_connection)
output("This is my first script mod - presented by Fufu508!!!")
output("Made using PyCharm.")
foo = 0
f = open("c:/tmp/proto-out.txt", "w+")
for num in range(1,10):
output(f'{num} is the number.')
f.write(f"writing line {foo}\n")
foo = foo + 1
if foo > 5:
output(f"foo is {foo}. We're done.")
break
f.close() |
|
|
|
Post by fufu508 on Feb 10, 2019 11:42:04 GMT -5
Hi andrew , unfortunately the "with" statement didn't seem to address the crash. I changed this: resource: io.BytesIO = resource_loader.load()
To this:
with resource_loader.load() as resource:
The console outputs the same as before, and shortly after that the game suddenly closes.
I'm wondering if there's something in the content of one or more of the keys that doesn't like to be read by the means we are using.
Now that I can write stuff to file, perhaps there are debugging statements I could add?
In the meantime, I will see if I can get the whole list of clips written to file before the game crashes. Crude but it will allow me to work with the list in my next steps.
|
|
|
|
Post by fufu508 on Feb 10, 2019 12:31:43 GMT -5
I now have the list of clips! Unfortunately it comes at the cost of crashing the game, until the cause can be found.
|
|
|
|
Post by fufu508 on Feb 10, 2019 13:33:26 GMT -5
In the spoiler is the current content of clips2file.py import io
import struct
import sims4.resources
import sims4.commands
def get_clip_name(resource: io.BytesIO) -> str:
resource.seek(0x38, io.SEEK_SET)
name_length: int = struct.unpack('i', resource.read(4))[0]
return resource.read(name_length).decode('ascii')
@sims4.commands.Command('clips2file', command_type=sims4.commands.CommandType.Live)
def clips2file(_connection=None):
output = sims4.commands.CheatOutput(_connection)
badkeys = 0
key = 0
clip_names_file = 'c:/tmp/ts4-anim-clips.txt'
clip_names :str = []
output('Gathering list of clips...')
keys = sims4.resources.list(type=sims4.resources.Types.CLIP_HEADER)
output(f'{len(keys)} clips found.')
for key in range(len(keys)):
try:
resource_loader = sims4.resources.ResourceLoader(keys[key], sims4.resources.Types.CLIP_HEADER)
with resource_loader.load() as resource:
clip_names.append(get_clip_name(resource))
except:
badkeys = badkeys + 1
output(f'Number of clips with keys that failed to read: {badkeys}')
output(f'Sorting the {len(clip_names)} clip names.')
clip_names.sort()
output(f'Writing the clip names to {clip_names_file}')
f = open(clip_names_file, 'w+')
for i in range (len(clip_names)):
f.write(f'{"{:5d}".format(i)} {clip_names[i]}\n')
f.close()
output(f'Done writing {clip_names_file}')
In case the script is consuming info from the game that it shouldn't, here's the current output with some custom clips in it. |
|
|
|
Post by zembee on Feb 21, 2019 13:20:40 GMT -5
Again thanks Andrew for the nice environment to create mods for the Sims 4. I just created a real mod that actually does something in the game, set days remaining before aging up, but there's going to be a new game patch released here soon, so I would think that the EA folder will need to be updated, therefore will just re-running decompile_all.py do the trick followed by recompiling the mod? Thanks!
|
|
|
|
Post by andrew on Feb 26, 2019 21:41:05 GMT -5
fufu508 I haven't given up on this. Can't figure out why the game crashes but may have another way  zembee glad you found this helpful. You can run decompile_all after any patch to update your copy of EA scripts. Once done, you can check your mods to see if EA has changed anything that you were using and recompile any that required changes to your script. If your script did not need to change and the version of Python didn't change, you probably won't need to recompile.
|
|
|
|
Post by andrew on Mar 1, 2019 17:52:55 GMT -5
fufu508 I tried adjusting the script so that it reads the package files directly instead of going through the sims4.resources module and had some success. Pros: 1. It can tell if a clip came from the mods folder since it loads those separately. 2. It doesn't crash.
Cons:
1. Possibly slower (didn't time it).
2. A lot more code involved. Try this and see if it gets you what you need:
import fnmatch
import io
import os
import os.path
import struct
import zlib
import sims4.resources
import sims4.commands
import sims4.log
import paths
from typing import List, Iterator, Tuple, Callable, Set
logger = sims4.log.Logger('Clip Test')
def read_package(filename: str, type_filter: set) -> Iterator[Tuple[int, int, int, Callable[[], bytes]]]:
type_filter = set() if not type_filter else type_filter
with open(filename, 'rb') as stream:
def u32() -> int:
return struct.unpack('I', stream.read(4))[0]
tag = stream.read(4).decode('ascii')
assert tag == 'DBPF'
stream.seek(32, io.SEEK_CUR)
index_entry_count = u32()
stream.seek(24, io.SEEK_CUR)
index_offset = u32()
stream.seek(index_offset, io.SEEK_SET)
index_flags: int = u32()
static_t: int = u32() if index_flags & 0x1 else 0
static_g: int = u32() if index_flags & 0x2 else 0
static_i: int = u32() << 32 if index_flags & 0x4 else 0
static_i |= u32() if index_flags & 0x8 else 0
for _ in range(index_entry_count):
t = static_t if index_flags & 0x1 else u32()
g = static_g if index_flags & 0x2 else u32()
instance_hi = static_i >> 32 if index_flags & 0x4 else u32()
instance_lo = static_i & 0xFFFFFFFF if index_flags & 0x8 else u32()
i = (instance_hi << 32) + instance_lo
offset: int = u32()
sz: int = u32()
file_size: int = sz & 0x7FFFFFFF
stream.seek(4, io.SEEK_CUR)
compressed: bool = sz & 0x80000000 > 0
compression_type: int = 0
if compressed:
compression_type = struct.unpack('H', stream.read(2))[0]
stream.seek(2, io.SEEK_CUR)
if compression_type not in (0x0000, 0x5A42):
continue
def load_func() -> bytes:
pos = stream.tell()
stream.seek(offset, io.SEEK_SET)
data = stream.read(file_size)
stream.seek(pos, io.SEEK_SET)
return zlib.decompress(data) if compression_type == 0x5A42 else data
if len(type_filter) == 0 or t in type_filter:
yield t, g, i, load_func
def get_clips_from_folder(folder: str) -> Tuple[Set[str], int]:
folder_clips: Set[str] = set()
pattern = '*.package'
failures: int = 0
for root, dirs, files in os.walk(folder):
for filename in fnmatch.filter(files, pattern):
package_path = str(os.path.join(root, filename))
try:
package_clips, package_failures = get_clips_from_package(package_path)
folder_clips |= package_clips
failures += package_failures
except Exception as ex:
logger.exception(f'Failed to read package {package_path}', exc=ex)
failures += 1
return folder_clips, failures
def get_clips_from_package(p) -> Tuple[Set[str], int]:
package_clips: Set[str] = set()
failures: int = 0
for t, g, i, load_func in read_package(p, type_filter={sims4.resources.Types.CLIP_HEADER}):
try:
with io.BytesIO(load_func()) as resource:
package_clips.add(get_clip_name(resource))
except Exception as ex:
logger.exception(f'Bad key: {t:X}:{g:X}:{i:X}', exc=ex)
failures += 1
return package_clips, failures
def get_clip_name(resource: io.BytesIO) -> str:
resource.seek(0x38, io.SEEK_SET)
name_length: int = struct.unpack('i', resource.read(4))[0]
name: str = resource.read(name_length).decode('ascii')
return name
@sims4.commands.Command('clips2file', command_type=sims4.commands.CommandType.Live)
def clips2file(_connection=None):
output = sims4.commands.CheatOutput(_connection)
clip_names_file = 'c:/tmp/ts4-anim-clips.txt'
mods_folder = os.path.expanduser(
os.path.join('~', 'Documents', 'Electronic Arts', 'The Sims 4', 'Mods')
)
game_folder = os.path.dirname(os.path.dirname(os.path.dirname(paths.APP_ROOT)))
game_clips, game_failures = get_clips_from_folder(game_folder)
mods_clips, mods_failures = get_clips_from_folder(mods_folder)
clip_set = game_clips | mods_clips
bad_keys: int = game_failures + mods_failures
output(f'{len(clip_set)} clips found.')
output(f'Number of clips with keys that failed to read: {bad_keys}')
output(f'Sorting the {len(clip_set)} clip names.')
clip_names: List[str] = list(clip_set)
clip_names.sort()
output(f'Writing the clip names to {clip_names_file}')
with open(clip_names_file, 'w+') as f:
for index in range(len(clip_names)):
f.write(f'{"{:5d}".format(index)} {clip_names[index]}\n')
output(f'Done writing {clip_names_file}')
|
|
|
|
Post by fufu508 on Mar 1, 2019 22:30:45 GMT -5
Thank you, andrew! I'll give it a try next time at my computer. |
|
|
|
Post by bloodyrose on Mar 31, 2019 18:36:56 GMT -5
I tried decompiling, but it doesn't do anything? There's no error, it just isn't getting the files. imgur.com/4ad3VvT |
|
|
|
Post by ts4playerfan on Apr 3, 2019 7:02:12 GMT -5
Hello, at the end of step 7 of "creating your own mod" I don't understand how to modify the python file to create my own mod.
|
|
|
|
Post by andrew on Apr 8, 2019 19:50:33 GMT -5
bloodyrose it looks like you may be on a Mac which the tutorial files aren't currently setup to support without modification. Try the post quoted below for the Mac differences. I made some changes to the code in settings.py and decomipile_all.py to get this to work under macOS (OSX) 10.13.6 (High Sierra). In settings.py: """ change game_folder for mac location """ game_folder = os.path.join('/', 'Applications', 'The Sims 4.app', 'Contents') In decompile.py: """ change gameplay_folder_game for mac location """ gameplay_folder_game = os.path.join(game_folder, 'Python') After the changes, the process went smoothly and was able to get the example to print out in the game. I'm using PyCharm 2018-2.4 with Python 3.7.1 with a Sims 4 install via Origin. I do have the prompt to upgrade to Mojave, but haven't yet, when I do I'll do it again if one is interested. Thanks Andrew for this step by step setup. I always got turned off at the de-compiling step. Sweet. this tutorial is very basic and only shows how to get python scripts into the game. In order to create your own mod, you will need to create and modify .py files in the script folder created in that section of the tutorial. What changes you make and what python files you create are entirely dependent on the mod that you are making. As a simple test, you could rename the python file copied from the example to something new, and rename the cheat code name in the python file and modify it to do something different instead of showing the text you entered when doing the tutorial. If everything worked correctly, you should have both the mod you made in the tutorial, and the new one working in the game at the same time.
|
|
