How should we implement in app purchases?

Yep - there are some threads about this already but I haven’t really found proper information on how to implement in app purchases. Choices made now are effected by the future of IAP provided by Unreal but I haven’t found a roadmap for that yet.

This question is not intended to evoke a list of “this thread has some information”. I’m looking for information on the future so developers can make informed choices about they approach this and what they expect from Epic.

This is obviously a major topic and I’ve tried to summarise my main questions and information I have so far below:

State of play at the moment:

  • IOnlineStore specialized for iOS and GooglePlay, as a wrapper around the basic high level management (shop interface) of dowloadable assets but NOT providing a way to download any actual assets which is left as a task for the individual teams.
  • IOnlineStore allows you to have users make purchases using the familiar platform handled purchasing system
  • For iOS you need to modify you defaultengine.ini to enable in app purchases and set your platform online subsystem, enable the GameCenter in your project settings and dynamically add the “DynamicallyLoadedModuleNames.Add(“OnlineSubsystemIOS”);”
  • I haven’t approached Android yet but it’s seems similar.
  • There is no support for XBox, Ps4, PC or MAC

Summary - from what I’ve seen - out of the box support exists for paid content that ships with the app, removing ads, subscribing to enable multiplayer features, enabling existing features.

Better than nothing, but leaving a lot of common work to be done by the developer teams using Unreal. IAP are going to be the bread and butter of one of the products I’m working on and highly likely where the majority of money made using Unreal will end up flowing. It would be good to know what the future holds or what the general advice is on how to proceed developing unreal apps with this as a long term plan.

Have others implemented their own download code and used the online store as a front end to that data?

The general questions I see (for my current needs and beyond) are:

  • Are package downloads planned - or implemented and I haven’t found them yet :wink:
  • What platforms are expected to get out of the box IOnlineStore support? Given the similarities with XBox and Playstation I would expect something there…
  • Does unreal have plans to partner with any of the third party store interfaces out there? It seems like something like Shopify could work either as a one shop stop or to fill in the gaps where there are not established store mechanisms.
  • Will Unreal host their own distribution system for end users (you’ve done the initial work in the developer marketplace right?)

For any “yes” responses even vaguely considered timelines would be pretty interesting.

What other features are people looking for in their IAP or DLC developer experience?

For what its worth my current plan is to:

  1. Build a front end experience based on IOnlineStore
  2. Package a bit of extra data (namely icons) into my app updates so I can make the store look pretty - if I care
  3. Develop initially with the data built into the main app, but enabled via IOnlineShop and engineered as much as possible like true DLC.
  4. Later implement specific iOS and Android download services saving the resultant files using the existing file systems.

On top of the content browsing and downloading we’re also going to need to package our content appropriately and distribute - leaving distribution as part of the Shop questions above what follows is the initial info I found on packaging.

Starting to look into this process:

[Edit] The “UnrealFrontend” seems useful: Using the Unreal Frontend Tool | Unreal Engine 5.1 Documentation

[Edit2] The Project Launcher with cooking options is standalone under DeveloperTools in the editor.

From the 4.7 release notes:

"Downloadable Content (DLC) Packaging *BETA*

Downloadable content (DLC) can now be packaged up for distribution!

Currently this feature is in beta testing and requires a command-line tool. Create your add-on DLC inside a game plugin Content folder, and use the new packaging tool to "cook" and optimize your content for your platform of choice.

This feature makes use of the game's asset registry to know what content is already supplied in the shipped game, and will cook anything needed from the game's content into the DLC.

To use this feature, run the cook utility for your main game with "-CreateReleaseVersion=x.y", then again for each DLC plugin with "-BasedOnReleaseVersion=x.y -DLCName=MyPluginName".

From 4.8 Release notes:

New: Downloadable Content (DLC) Support

Added support for cooking DLC (Downloadable Content). This allows you to create Pak files that can be distributed to users later.
Games can choose to make sure DLC doesn't reference Engine content (that Epic will be changing over time).
In Unreal Frontend, make a custom profile, then select Cook -> Release / DLC / Patch Settings -> Include Engine Content.

From 4.9 Release Notes:

Cooking dlc no longer includes assets from base game in the asset registry.

Stumbling block #1 was needing the -BasedOnReleaseVersion=0.1 flag in the cooker settings for the DLC profile.

Somewhere in proceedings I also setup a profile for cooking my project with a version 0.1 - so I’m not sure if this is needed or not.

Now the cooker appears to do some work, though it ends with an error on launch and doesn’t actually seem to create a .pak file (small issues … )

Stumbling block #2 I’m stuck on right now - trying to work out how to get the DLC builder to package loose content (json files etc) within the DLC structure. For the main game this is done in the package settings. I’m also not sure how to author any content within the DLC directory in my Plugins subfolder.

Well what fun this is!!

I didn’t get very far with building DLC - it wasn’t apparent how I could put assets in the plugin directly (the content system complained about the path when I tried to use it) so I’ve done some experimentation into using UnrealPak and pak files.

To this end I’ve managed to inject a json file into a pak file and then load it in game. It’s not working as I’d like and documentation is thin on the ground so I’m going to describe the process.

I’m still very unsure how to build unreal content to go in a pak file and not get packaged into the main game, but right now that’s a topic for next week and the rest of this week will be spent making shiny things.

Comments below include the major things I’ve learnt.

Building a pak file.

set "UNPACKDIR="C:\Program Files\Unreal Engine\4.9\Engine\Binaries\Win64\""
%UNPACKDIR%\UnrealPak.exe "%cd%"\DLCPackage1.upack -create="%cd%"\filelist.txt

Contents of filelist.txt:

"C:\Users\\Documents\Unreal Projects\MM\Plugins\DLCPackage1\Content\*.*"

I then put the upack file in a top level folder in my project.

gotcha:

I did start putting the upack into a folder inside my content directory and the editor started doing things automatically, namely extracting files into the Content directory where source control started trying to add them despite them living somewhere else. It was a mess. Outside the Content directory works. Also - I wasn’t sure how to surpess packaging if they live in the content folder.

Mounting a packfile

This took more time and I’m not happy with the results.

I have multiple DLC pak files, and mount a single FPakPlatformFile. The bit that’s been taking time is the mount point. Stepping into the file reading code it seems like the system trys to path into the engine directory structure. Reading around in the forums this has tripped up others. The way around it seems to be to treat it like a drive. See the mount point below!

bool DLCManager::MountPak(const FString &fullPath)
{
	FString MountPoint = TEXT("d:");

	//Load and mount pak file
	if (mp_DLCPakFiles == nullptr) {
		mp_DLCPakFiles = new FPakPlatformFile;
		if (!mp_DLCPakFiles->Initialize(&FPlatformFileManager::Get().GetPlatformFile(), TEXT(""))) {
			delete mp_DLCPakFiles;
			return false;
		}
	}

	return mp_DLCPakFiles->Mount(*fullPath, 0, *MountPoint);
}

gotcha:

I’ve lost the Content directory I packed the pak file with. Right now this is enough to get me going but there’s considerable work left to do. My file path to load a file originally in DLCPackage1\Content is d:\MyFile.json.

Opening a file

This part is fairly easy:

mp_DLCPakFiles->OpenRead(pFileName)

However I’m pretty sure that it’s possible to mount mp_DLCPakFiles into the file system so you don’t need to special case loading the data (see below).

Reading a file

So - I was using FFileHelpers to read a file as a string. Now we need to access mp_DLCPakFiles (see above for wanting to get around special case loading) to load a file this doesn’t work. The code below manages loading a json file - the string is only needed if your file contains special characters otherwise you can create the json factory with a simple memory archive and avoid a load of memory allocating and copying.

        auto handle = FPlatformFileManager::Get().GetPlatformFile().OpenRead(*rDescFileName, false);
		if (handle == nullptr){
			//Try loading from DLC!!
			handle = DLCManager::GetInstance().OpenRead(*rDescFileName);
			if (handle == nullptr)
				return false;
		}

		TArray<uint8> dest;
		auto size = handle->Size();
		dest.Reset(size);
		dest.SetNum(size);
		bool bResult = handle->Read(dest.GetData(), size);
		delete handle;
		if (!bResult)
			return false; 

		FString tmp;
		FFileHelper::BufferToString(tmp, dest.GetData(), size); //This handles special characters
		TSharedRef < TJsonReader<> > jsonReader = TJsonReaderFactory<>::Create(tmp);

Major Questions:

  • [Edit - Resolved]Directory structure
  • [Edit - Resolved]Integrating into file system
  • Cooking
  • Adding unreal data without it duplicating in the Content directory / final packed game
  • [Edit - Resolved]*Actually downloading the data from somewhere cool

Integrating into file system

I thought I’d tried this before but looks like I put it in the wrong place in my code and was having other issues preventing me from seeing it work.

FPlatformFileManager::Get().SetPlatformFile(*mp_DLCPakFiles);

As mp_DLCPakFiles sets up the original platform file as its Inner (fallback) file system we now have a one shop file loading system with newly mounted packfiles taking precedence over old files. If only the directory system was working with what I’ve done to get this far…

gotcha

If you’re going to do this and delete your mp_DLCPakFiles equivalent you’ll need to restore the file system (not sure why it doesn’t do this internally).

	if (mp_DLCPakFiles->GetLowerLevel())
	{
		FPlatformFileManager::Get().SetPlatformFile(*mp_DLCPakFiles->GetLowerLevel());
	}
	delete mp_DLCPakFiles;
	mp_DLCPakFiles = nullptr;

Directory Structure

[Edited!!!]

For directory structure I’ve managed to get sane reading paths without the d: hack I was using previously.

Pak files seem to store paths as an offset from the pack file location at time of build. You specify the mount point when you mount the pak file.

NOTE! This only started working for me after using FPaths::MakeStandardFilename on the mount point, and watch out for \ vs / when trying to load files, you’ll want /

mp_DLCPakFiles->Mount(*fullPath, 0, FPaths::MakeStandardFilename(pointInPathTheUpackFileOriginalLocationMapsTo));

This is good progress folks.

One question remains regarding directory structure though - how do I build unreal content for these pak files. There seem to be two likely approaches:

  1. Get the content system to allow adding assets outside the content directory
  2. Get the packaging system to avoid certain directories within Content while packaging the game.

Regarding actually downloading - I’ve started looking into apple hosted content. They provide an interface for this however the first potential blocker is that the purchase transaction (hidden inside the shop) contains the link for the download.

“When the user purchases a product that has associated Apple-hosted content, the transaction passed to your transaction queue observer also includes an instance of SKDownload that lets you download the associated content.”

More to work out here!

Here’s some documentation.

Could you share a test project?

A test project would be lovely - especially if it incorporated the actual data download from any hosts that already support it :slight_smile:

I won’t be making one myself, don’t have the time to make anything meaningful, kids to feed and a roof to fix so to speak.

However I’ll be back in purchase code within he next two weeks and will add any further info I get to this thread. Also if you get going and have specific problems I’m happy to have a go at helping out (the posts above represent clues to try and shorten the learning curve).

Hi, I’m trying to create .pak files recently, but i’m stuck now. would you mind sharing more details about it? I

Sorry for the late reply - what are you getting stuck on?

Actually downloading the data from somewhere cool

Spent a bit of time on this. After some humming and harring I ended up signing up for Amazon AWS. They have a years free development and friends in the know recommend them.

A conversation with Moss and joeGraff consolidated the idea of downloading via http and off I set…

It proved easy to set up a amazon s3 bucket and copy a file to it. Downloading via the Unreal Http system was fairly straightforward too but when I added permissions I got stuck. Turns out HTTP signing for Amazon
is quite tricky and the examples I found out there didn’t map well to core Unreal functionality.

After 24 hours of sweat though I have something working - and can download a signed file using the Authorization ID and secret Key.

###AWSS3HttpDownloader
AWSS3HttpDownloader does the brunt of the work. Its a singleton class and uses the Http module (you’ll need to add to your build.cs for it to link).

[EDIT: So far tested in IOS and PC with a variety of files including pack files with raw image files and json included - my files have progressed from those posted but the core code remains the same]

In the zip attached there are also functions for SHA256, and HMAC SHA256 hashing and key generation.

###Example use:

 mp_HttpDownloader = new AWSS3HttpDownloader();
         if (!mp_HttpDownloader->StartService())
         {
             Errorf(_T("Failed to start Http Download service"));
         }
         else
         {
             AWSS3Credentials credentials(AuthID, AuthKey);
             mp_HttpDownloader->StartDownload(credentials, "/mybucket", "/MarketPlaceData/DLCDirectory.mdc", OnDLCDescriptionDownload);
         }

You put your post file loaded (or Error - check the response code and act appropriately) functionality in whatever callback you define for OnDLCDescriptionDownload.

At the moment I’m looking at downloading user generated content from amazon. Text is straightforward, the code below loads a PNG file on iphone, PC and MAC (editor and non editor builds - have yet to try shipping).

Audio is proving a bit trickier, WAV loading works in the editor but not the standalone game.

UTexture* Data_RawImage::Load(const FString &fullName) const
{
	if (!FPaths::FileExists(fullName))
	{
		return nullptr;
	}

	FString ext = FPaths::GetExtension(fullName);
	EImageFormat::Type fmt = EImageFormat::Invalid;
	if (ext.Contains("jpg"))
	{
		fmt = EImageFormat::JPEG;
	}
	else if (ext.Contains("png"))
	{
		fmt = EImageFormat::PNG;
	}
	else
	{
		return nullptr;
	}

	TArray<uint8> RawFileData;
	if (!FFileHelper::LoadFileToArray(RawFileData, *fullName))
	{
		return nullptr;
	}

	IImageWrapperModule& ImageWrapperModule = FModuleManager::LoadModuleChecked<IImageWrapperModule>(FName("ImageWrapper"));
	IImageWrapperPtr ImageWrapper = ImageWrapperModule.CreateImageWrapper(fmt);
	if (!ImageWrapper.IsValid())
	{
		return nullptr;
	}

	if (!ImageWrapper->SetCompressed(RawFileData.GetData(), RawFileData.Num()))
	{
		return nullptr;
	}

	const TArray<uint8>* UncompressedBGRA = NULL;
	if (ImageWrapper->GetRaw(ERGBFormat::BGRA, 8, UncompressedBGRA))
	{
		auto *pNewTexture =  UTexture2D::CreateTransient(ImageWrapper->GetWidth(), ImageWrapper->GetHeight(), PF_B8G8R8A8);
		if (pNewTexture != nullptr)
		{
			void* TextureData = pNewTexture->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_WRITE);
			FMemory::Memcpy(TextureData, UncompressedBGRA->GetData(), UncompressedBGRA->Num());
			pNewTexture->PlatformData->Mips[0].BulkData.Unlock();
			pNewTexture->UpdateResource();
			return pNewTexture;
		}
	}

	return nullptr;
}