x

Search in
Sort by:

Question Status:

Search help

  • Simple searches use one or more words. Separate the words with spaces (cat dog) to search cat,dog or both. Separate the words with plus signs (cat +dog) to search for items that may contain cat but must contain dog.
  • You can further refine your search on the search results page, where you can search by keywords, author, topic. These can be combined with each other. Examples
    • cat dog --matches anything with cat,dog or both
    • cat +dog --searches for cat +dog where dog is a mandatory term
    • cat -dog -- searches for cat excluding any result containing dog
    • [cats] —will restrict your search to results with topic named "cats"
    • [cats] [dogs] —will restrict your search to results with both topics, "cats", and "dogs"

Swarm - What is it and how to use it?

So from usage and Google searches re UDK I am guessing that Swarm is a program within the Unreal4 engine that manages mulitple clients to undertake distributed compilation of primarily Lightmass solutions but possible other aspects of maps.

My questions for you fine folks are:

1) Am I correct above in describing Swarm or does it do something differently than this?

2) Is there documentation available for its set up or operation?

3) Do I need to install the full Unreal Engine on the distributed clients or is there a sub-set I can use (what is the minimum client to place on slaves?)?

4) [Unlikely I know but...] Is there Linux version of the client? If not, has anyone had success using a windows VM under Linux?

5) To start the client do I just run the "SwarmAgent.exe" found on my Windows Box at C:\Program Files\Unreal Engine\4.0\Engine\Binaries\DotNET ?

Product Version: Not Selected
Tags:
more ▼

asked Apr 10 '14 at 12:02 PM in Using UE4

avatar image

Dingo_aus
11 3 4 4

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

2 answers: sort voted first

So from usage and Google searches re UDK I am guessing that Swarm is a program within the Unreal4 engine that manages mulitple clients to undertake distributed compilation of primarily Lightmass solutions but possible other aspects of maps. {..} 1) Am I correct above in describing Swarm or does it do something differently than this?

Yes, Swarm (short for UnrealSwarm) is a standalone C# application that distributes units of work across multiple machines on the network. Currently, Swarm is only used in conjunction with Lightmass to pre-compute lighting for levels, but the application itself is task agnostic, which means that it can work with any other type of task as well. We just haven't taken advantage of that yet.

2) Is there documentation available for its set up or operation?

UnrealSwarm was created for Unreal Engine 3, and it looks like the documentation hasn't made it into the UE4 Docs web site yet. That being said, Swarm hasn't really changed (except for a few bug fixes), so the old documentation still applies. You can find it here:

http://udn.epicgames.com/Three/Swarm.html

http://udn.epicgames.com/Three/Devel...#Swarm%20Setup

3) Do I need to install the full Unreal Engine on the distributed clients or is there a sub-set I can use (what is the minimum client to place on slaves?)?

No, you do not need to install Unreal Engine on the Swarm agent machines. Swarm will automatically transfer all files (including binaries) that are needed to process a given job. The easiest way is to copy the /Engine/Binaries/DotNET/ directory from an existing UE4 installation to a new computer. Then follow the instructions on the pages linked above.

4) [Unlikely I know but...] Is there Linux version of the client? If not, has anyone had success using a windows VM under Linux?

UnrealSwarm currently only runs on Windows operating systems. You may be able to get it to run in a Windows emulator on Linux. We briefly had a version that worked on MacOS, but newer versions of Mono introduced some breaking changes that we haven't fixed yet. We are planning to rewrite UnrealSwarm in C++, and at that point it will also become cross-platform and seamlessly work on Linux and MacOS. There is no timeline for this project yet, but it will probably happen this year - possibly as early as this Summer.

5) To start the client do I just run the "SwarmAgent.exe" found on my Windows Box at C:\Program Files\Unreal Engine\4.0\Engine\Binaries\DotNET ?

Yes.

I can't answer #5 yet because when I execute the executable I get a "Register for Visual Studio C++ dialog box" - Made me laugh that an executable needs registration with Microsfot to run, it has been a long time since using Windows

You may be missing some prerequisites for UnrealSwarm. Make sure that the latest version of the .NET Framework is installed on your computer. You may also need the Visual Studio redistributable package. The easiest way to install prerequisites is to install /Engine/Extras/Redist/en-us/UE4PrereqSetup.exe

more ▼

answered Apr 11 '14 at 05:42 PM

avatar image

gmpreussner STAFF
4.2k 124 55 228

avatar image Ellert van Koperen Dec 09 '14 at 03:04 PM

How much data is actually transferred at the startup of swarm to the slaves ? Depending on this, it might or might not be interesting to offload lightmass cooking to another site (assume a vpn connection), or even a cloud. No sure if it is possible, but Azure springs to mind, and if the lightmass agent and swarm is ever improved to a linux version, any cloud would theoretically be able to offload this heavy process.

avatar image Jarek STAFF Dec 15 '14 at 08:00 AM

The UnrealSwarm usage itself is rather neglectible. The problem would be with Lightmass data. It highly depends on the level, but it may rise up to several Gb for large levels.

You can find out yourself how much data is there looking into the Swarm job folder in swarm cache for your concrete level.

So it looks like there is currently no way of transfering this data through the Internet (depends on bandwidth of course :)). At least it wasn't designed for that.

avatar image Darkflame Dec 21 '14 at 01:53 PM

I'm assuming the swarm computers do not require a GPU?

avatar image Darkflame Dec 21 '14 at 03:11 PM

Also, how much ram would a swarm pc require?

Assuming 4gb will be fine since it doesn't require the Unreal Engine to be running?

avatar image Jarek STAFF Dec 26 '14 at 08:59 AM

Hi Darkflame, this is more Lightmass question rather than Swarm, cause Swarm only copies and deploys Lightmass tasks to remote machines.

Lightmass does its computation on CPU only. Its highly multithreaded, so the more cores the better. It is also recommended to have 1 Gb of RAM per thread (at least for big scenes). You'll need to have WinXP or higher. There is also a strong recommendation for 64-bit system as it's 30% faster and you won't hit the out-of-memory exception if you install on that machine more than 4 Gb.

Sorry for the long response, but we have X-mass break :).

Jarek

avatar image devel.bmad Jul 13 '16 at 03:52 PM

Has this changed since Apr 2014?

Specifically: is SwarmAgent available on mac now and is it possible to distribute the lightmass calculations from mac to remote PCs?

avatar image ChrisBradwell Oct 27 '16 at 09:22 PM

@devel.bmad strange isn't it, many many months later and it seems no one else wants to distrubute lighting rendering.

(comments are locked)
10|2000 characters needed characters left
Viewable by all users
  1. That is correct, it is used to distribute lightmass. (And nothing else)

  2. If documentation is still limited its operation and interface seem to be identical to the ue3 version so you could also look at documentation written for ue3/udk.

  3. You probably won't need everything, although I'm not sure which part that would be.

  4. No, and no idea.

  5. That is correct, although you also need one coordinator running on your network to manage the different agents.

more ▼

answered Apr 10 '14 at 01:22 PM

avatar image

Arnage
858 22 16 52

(comments are locked)
10|2000 characters needed characters left
Viewable by all users
Your answer
toggle preview:

Up to 5 attachments (including images) can be used with a maximum of 5.2 MB each and 5.2 MB total.

Follow this question

Once you sign in you will be able to subscribe for any updates here

Answers to this question