Import .Pak from remote server

Hello,

I’m currently trying to load a mesh from a server. So, I thought about using Pak files.
Now, I want to download a Pak from a server and load assets that are in the Pak file.

I didn’t find help or tutorials anywhere, except https://answers.unrealengine.com/questions/109485/stream-an-asset-from-the-internet.html that helped me a lot but not enough to do what I wanted.
From theses informations, I’m not sure to understand the using of FPlatformFileManager/IPlatformFile and the mount system (for PakFile/PakPlatform) stuff, and what should I do after I loaded the PakFile.

TSharedPtr<FStreamingNetworkPlatformFile> NetPlatform = new FStreamingNetworkPlatformFile();

if (NetPlatform.IsValid())
{
	NetPlatform->Initialize(LocalPlatformFile, *FString::Printf(TEXT("-FileHostIP=%s"), *ServerHost)); // ServerHost = "127.0.0.1:41899" - The UnrealFileServer port
	return false;
}

FPakFile PakFile(NetPlatform.Get(), TEXT("OutPak.pak"), false);

if (!PakFile.IsValid())
	return false;

TSet<FString> FileList;
PakFile.FindFilesAtPath(FileList, *PakFile.GetMountPoint(), true, false, true); // The mount point is at "/"

StreamedAssets.Empty();
for (TSet<FString>::TConstIterator FileItem(FileList); FileItem; ++FileItem)
{
	FString AssetName = *FileItem;
	if (AssetName.EndsWith(FPackageName::GetAssetPackageExtension()) ||
		AssetName.EndsWith(FPackageName::GetMapPackageExtension()))
	{
		StreamedAssets.Add(AssetName); // Just one item : "lit.uasset" -- beside, that should be "lit.uasset", but that means that my .Pak has been loaded correctly from the server since that name and the size of the uasset are corrects
	}
}

FPlatformFileManager::Get().SetPlatformFile(*NetPlatform); // I think I had to do that so say to the StreamableManager : "Hey, load from the NetPlatform, to load the file from the remote server" -- I have no idea it's really useful or not
UObject *object = StreamableManager->SynchronousLoad(StreamedAssets[0]); // Of course, object is null after that since it's all wrong

Beside, what kind of path should take SynchronousLoad/LoadObject/StaticLoadObject ?
Also, does someone has more informations about the using of the UnrealFileServer ? Couldn’t find anything about it.

I will take any help or clue :slight_smile:
Thanks for reading.

Did you ever find a solution? I have the same problem but I can’t find anything very helpful besides the same link you posted.

I also looked into this.

The demo shooter game supposedly does something along the lines of what you’re looking for. I could barely make sense of it(I don’t know C++ very well) Maybe you’ll have a better time.

Unfortunately, I couldn’t find a way to do it.

However, maybe you can find a way with the 4.8. They added a DLC system (Unreal Engine 4.8 Release Notes | Unreal Engine Documentation - just search for ‘DLC’) but I couldn’t find more informations about how to you load it (except the using of the command line to start the program with the Pak file loaded)…

I’m going to check the demo shooter game and I will tell you if I find anything.

Edit : Just by checking the names of the C++ classes, I don’t think there is something relative to Pak files. To be sure that I’m checking the good project, is it the futuristic shooter where there is some flying cars outside and a city with some big buildings ? Or maybe can you provide me more informations about which direction I should look (is it to add in runtime new weapons ? new items ? new game modes ?)

For me, the options “Generate Chunks and Build Http Chunk Install Data” causes an error during the build, so I couldn’t test and when without theses options, the game crashes a few seconds after the start, during the video…

However, I checked the code and not really sure of what it does… But it seems very relative to PS4 : Can I get more info on what the "ShooterGameDelegates.cpp" does - Programming & Scripting - Unreal Engine Forums
There is a system on the PS4 called “PlayGo” that, if I understood correctly, allow the player to play while the game is installing. The installation is cut in “chunks” but I’m not sure that they are .Pak.

In the function AssignStreamingChunk (ShooterGameDelegates.cpp), there is this comment : “Add assets to map paks unless they’re engine packages or have already been added to the base (engine) pak.”. I understand the code but I don’t understand how it adds a package from this.

Sorry for not being able to help more…

Thank you and good luck !

No problem. I’m not sure where else to look for an answer. I’ll let you know if I find one though.

Are you able to mount a .pak file that is stored locally in c++?

It’s been a long time so I’m not very sure but I believe I have not even managed to do that…

There are solutions to this problem, but no detailed information。

Can you help me ? thank you!

same issue here … need to get pak file from remote server via command line … here is my post, but no answer yet …

hi guys, somebody does implemented this stuff?

ok, I have some progress.

I deployed the local server using xampp (and place in the htdocs my chunks - which I set through asset labels).

first I download by http get file, then mount it and read the data inside it.

header

// Fill out your copyright notice in the Description page of Project Settings.

#pragma once

#include "CoreMinimal.h"
#include "GameFramework/Actor.h"
#include "Runtime/Online/HTTP/Public/Http.h"
#include "LoadAssets.generated.h"

UCLASS(config=TestConfigFile)
class TESTCPP_API ALoadAssets : public AActor
{
	GENERATED_BODY()
	
public:	
	// Sets default values for this actor's properties
	ALoadAssets();

protected:
	// Called when the game starts or when spawned
	virtual void BeginPlay() override;

public:	

	FHttpModule* Http;

	FString PakGamePath;
	FString ChunkName;
	FString FileSavePath;

	UPROPERTY(Config, BlueprintReadOnly) //temporary set in the TestConfigFile Name ChunkToDownload.
	FString ChunkToDownload;
	
	UFUNCTION(BlueprintCallable, Category = "LoadAssets") //runs from blueprint http://joxi.ru/Vrwo64MU75BqRr
	void DownloadAsset(const FString& AssetURL);

	void OnResponseReceived(FHttpRequestPtr Request, FHttpResponsePtr Response, bool bWasSuccessful);

	UFUNCTION()
	bool MountChunk();
};

cpp file

// Fill out your copyright notice in the Description page of Project Settings.


#include "LoadAssets.h"
#include "Engine/Engine.h"
#include "Misc/Paths.h"
#include "HAL/PlatformFilemanager.h"
#include "Misc/Paths.h"
#include "GenericPlatform/GenericPlatformFile.h"
#include "IPlatformFilePak.h"

// Sets default values
ALoadAssets::ALoadAssets()
{
	Http = &FHttpModule::Get();
}

// Called when the game starts or when spawned
void ALoadAssets::BeginPlay()
{
	Super::BeginPlay();
	
	PakGamePath = FPaths::ProjectContentDir() + TEXT("Paks/");	//UE_LOG(LogTemp, Warning, TEXT("ProjectContentDir() -- %s"), *PakGamePath);
}

void ALoadAssets::DownloadAsset(const FString& AssetURL)
{
	TSharedRef<IHttpRequest> HttpRequest = Http->CreateRequest();
	HttpRequest->SetVerb("GET");
	HttpRequest->SetURL(AssetURL);
	HttpRequest->OnProcessRequestComplete().BindUObject(this, &ALoadAssets::OnResponseReceived);
	HttpRequest->ProcessRequest();
}

void ALoadAssets::OnResponseReceived(FHttpRequestPtr Request, FHttpResponsePtr Response, bool bWasSuccessful)
{
	if (Response.IsValid() && EHttpResponseCodes::IsOk(Response->GetResponseCode()))
	{
		ChunkName = FPaths::GetCleanFilename(Response->GetURL());
		FileSavePath = PakGamePath + ChunkName; //UE_LOG(LogTemp, Warning, TEXT("FileSavePath -- %s"), *FileSavePath);
		IPlatformFile& PlatformFile = FPlatformFileManager::Get().GetPlatformFile();
		IFileHandle* FileHandle = PlatformFile.OpenWrite(*FileSavePath);
		if (FileHandle)
		{
			// Write the file
			FileHandle->Write(Response->GetContent().GetData(), Response->GetContentLength());
			// Close the file
			delete FileHandle;

			UE_LOG(LogTemp, Warning, TEXT("FileHandle exist, MountChunk()"));
			MountChunk();
		}
		else
		{
			UE_LOG(LogTemp, Warning, TEXT("FileHandle NOT exist, possibly chunk with the same name already is, or Name of chunk is missing"));
		}
	}
	else
	{
		UE_LOG(LogTemp, Warning, TEXT("Request failed"));
	}
}

bool ALoadAssets::MountChunk()
{
	IPlatformFile& PlatformFile = FPlatformFileManager::Get().GetPlatformFile();
	
	FPakFile PakFile(&PlatformFile, *FileSavePath, false);
	if (!PakFile.IsValid())
	{
		UE_LOG(LogTemp, Warning, TEXT("PakFile -- Not Valid"));
		return false;
	}
	
	FString MountPoint = PakGamePath; // + ChunkName //UE_LOG(LogTemp, Warning, TEXT("ChunkName: %s"), *ChunkName);
	PakFile.SetMountPoint(*MountPoint);

	UE_LOG(LogTemp, Warning, TEXT("PakFile.GetMountPoint(): %s"), *PakFile.GetMountPoint());

	TArray<FString> FileList;
	PakFile.FindFilesAtPath(FileList, *PakFile.GetMountPoint(), true, false, true);
	for (int32 it = 0; it < FileList.Num(); it++)
	{
		FString file = FileList[it];
		UE_LOG(LogTemp, Warning, TEXT("[UPackageDownloader::MountPackage] Content %s: %s"), *ChunkName, *file);
	}    
	return true; 	
}

after all you can get path of the assets inside package and for ex. spawn it.
just for example code (not bounding with above):

		PakFile.FindFilesAtPath(FileList, *MountPoint, true, false, true);
		for (int32 it = 0; it < FileList.Num(); it++)
		{
			FString AssetName = FileList[it];
			FString GamePathName, TestExtension;
			FPackageName::TryConvertFilenameToLongPackageName(AssetName, GamePathName);			
			FString AssetShortName = FPackageName::GetShortName(AssetName);
			FString FileName, FileExt;
			AssetShortName.Split(TEXT("."), &FileName, &FileExt);
			if (FileExt == "uexp") continue;
			else
			{				
			    FString NewAssetName = GamePathName + TEXT(".") + FileName;
				TestSpawnFromPath(NewAssetName);
				//UE_LOG(LogTemp, Warning, TEXT("AssetName -- %s, TestFileName -- %s, NewAssetName -- %s, AssetShortName -- %s"), *AssetName, *GamePathName, *NewAssetName, *AssetShortName);
			}
		}
	}	
}

void ATestActor::TestSpawnFromPath(FString Path)
{
	FSoftObjectPath StrNewAssetRef;
	if (true) ///blueprints
	{
		StrNewAssetRef = "Blueprint'" + Path + "_C'";
	}
	else
	{
		StrNewAssetRef = Path;
	}
	//UE_LOG(LogTemp, Warning, TEXT("StrNewAssetRef -- %s"), *StrNewAssetRef.ToString());
	UClass* SpawnClass = StaticLoadClass(UObject::StaticClass(), NULL, *StrNewAssetRef.ToString(), NULL, LOAD_None, NULL);
	if (SpawnClass != nullptr)
	{		
		//UE_LOG(LogTemp, Warning, TEXT("NewLoadedObject -- %s"), *SpawnClass->GetName());		
		Transform.SetLocation(FVector(Transform.GetLocation().X, Transform.GetLocation().Y + 300.f, Transform.GetLocation().Z));
		GetWorld()->SpawnActor<AActor>(SpawnClass, Transform);
	}
}
1 Like

I can’t check that right now but if you succeed to make it work, I think I can finally set it as an answer!
Great job, because the lack of documentation (or at least, 4 years ago) made the task really hard!
Thank you!

Hi 6rom, I tried this out today, and got downloading to work from the example, but not mounting. The MountChunk function seems to only set the mount point and list the files in the .pak. What should I expect to happen when running MountChunk()? Also does mounting only work in packaged game?

Hi, MagicBots.

thanks and sorry that I didn’t make a full test.
I fixed it by @theonecalledtom suggestion .
I attached archive with .h and .cpp , please check it.
link text

code isn’t clear and ideal cause I’m not very cool in cpp right now :frowning:
but I think it can give some help :wink:

ps: this will be working in packaged game only

Thank you for the file, and good to know it only works in packaged game. So after reading around i see that content/paks is supposed to be some automatic directory for auto mounting DLC… and so i realized this is why you were not manually mounting. Is there something i am supposed to set in packaging settings so that the content/paks directory works like it is supposed to for automatically mounting the .paks in this directory?

Nevermind. I see you are now manually mounting with the .zip you supplied and it works. This is great. thanks.