Search in
Sort by:

Question Status:

Search help

  • Simple searches use one or more words. Separate the words with spaces (cat dog) to search cat,dog or both. Separate the words with plus signs (cat +dog) to search for items that may contain cat but must contain dog.
  • You can further refine your search on the search results page, where you can search by keywords, author, topic. These can be combined with each other. Examples
    • cat dog --matches anything with cat,dog or both
    • cat +dog --searches for cat +dog where dog is a mandatory term
    • cat -dog -- searches for cat excluding any result containing dog
    • [cats] —will restrict your search to results with topic named "cats"
    • [cats] [dogs] —will restrict your search to results with both topics, "cats", and "dogs"

Trouble with an Oil Painted Post Effect

Hello folks. I'm trying to implement an algorithm I found for creating an oil panted effect over a given image:


but I can't seem to get the lil bugger to work. I'm using a custom node in a post process material to handle the nested FOR loops needed to compare the pixels surrounding the pixel being rendered, and everything seems to compile fine, but when I apply the effect in my scene, all I get is some weird noise.

alt text

So, obviously something is failing. I'm assuming it's the SceneTextureLookup call I am making within the custom node, but I can't find any decent docs or examples of how to use this function to be sure if that's the cause or not. If anyone is able/willing, could you please take a look at my material & code and point out what idiot thing I am doing, I would be very grateful.

Here's the material:

alt text

And here's the code in the OilPaint node:

 float4 outputBuffer = float4(1,0,1,1);
 float4 curPixel = float4(1,0,1,1);
 float nSumR[50];
 float nSumG[50];
 float nSumB[50];
 float intensityCount[50];
 for(int bob = 0;bob < 50;bob++)
     nSumR[bob] = 0.0;
     nSumG[bob] = 0.0;
     nSumB[bob] = 0.0;
     intensityCount[bob] = 0.0;
 for( float yRadInc = (curY - effectRadius); yRadInc <= (curY + effectRadius); yRadInc++ )
     if(!( (yRadInc < 0.0) || (yRadInc > height)))
         for( float xRadInc = (curX - effectRadius); xRadInc < (curX + effectRadius); xRadInc++ )
             if(!( (xRadInc < 0.0) || (xRadInc > width)))
                 curPixel = SceneTextureLookup(float2(xRadInc,yRadInc), 14, false);
                 float nR = curPixel.r;
                 float nG = curPixel.g;
                 float nB = curPixel.b;
                 // Find intensity of RGB value and apply intensity level.
                 float curIntensity =  ( ( ( nR + nG + nB ) / 3.0 ) * effectIntensity );
                 if( curIntensity > 1.0 )
                     curIntensity = 1.0;
                     int i = (int)curIntensity;
                 nSumR[i] = nSumR[i] + nR;
                 nSumG[i] = nSumG[i] + nG;
                 nSumB[i] = nSumB[i] + nB;
 float nCurMax = 1.0;
 int nMaxIndex = 0;
 for( int nI = 0; nI < (effectRadius * effectRadius); nI++ )
     if( intensityCount[nI] > nCurMax )
         nCurMax = intensityCount[nI];
         nMaxIndex = nI;
 outputBuffer = float4((nSumR[nMaxIndex] / nCurMax),(nSumG[nMaxIndex] / nCurMax),(nSumB[nMaxIndex] / nCurMax),1.0);
 return outputBuffer;

As you can see, I've done my best to duplicate the algorithm from the link at the top, but it's just not working. Hopefully it's some simple thing I'm over looking and not some intrinsic limitation to the post process material that is preventing this from working as I'd think it would look pretty cool...

Thanks in advance for any help.



Product Version: UE 4.12
more ▼

asked Jun 24 '16 at 12:15 PM in Rendering

avatar image

J. J. Franzen
6 1 4 4

avatar image J. J. Franzen Jun 27 '16 at 11:27 PM

To update this within info gleaned from posting in the forums, I have gotten the effect to work as a basic material after fixing some issues with the code, such as not using normalized UV coors, etc. but have hit a roadblock in that the SceneTextureLookup function refuses to run within nested loops. The compiler gives the error of:

Error [SM5] Material.usf(1283,14): error X4014: cannot have gradient operations inside loops with divergent flow control

If anyone knows of a way to work around this, I'd love to hear about it. It feels like I'm really close, but this seems to be an internal engine thing. Be a real bummer if I couldn't get this working as a PPE...

Here's the updated (working) graph and code for the material version:

alt text

link text

avatar image J. J. Franzen Jun 27 '16 at 11:29 PM

And 2000 words worth of proof:


alt text


alt text

beforeeffect.png (866.4 kB)
workingeffect.png (901.4 kB)
(comments are locked)
10|2000 characters needed characters left
Viewable by all users

1 answer: sort voted first

Error [SM5] Material.usf(1283,14): error X4014: cannot have gradient operations inside loops with divergent flow control

Hey this is a few months late, I'm just posting this for the next guy. If you get the above error in a For or If statement, change the way you're grabbing texture pixels. For example:

 [loop]  // This forces it to loop the For loop. To unravel it, you could use [unroll(8)]
 for (int i = StartIndex; i < EndIndex; i++)
     float V = floor(i / Width);
     float U = (i - (V * Width)) / Width;
     V = V / Width;
     float2 UV = { U,V };
     //BAD; This samples multiple MIP levels, which isn't allowed different length For loops
     //sum = sum + Texture2DSample(TEX, Material.Texture2D_0Sampler, UV);
     //GOOD; This only samples one MIP level, and thus loops fine
     sum = sum + Texture2DSampleLevel(TEX, Material.Texture2D_0Sampler, UV,MaterialFloat(0.00000000));

I got the texture sample code by going to >Material Editor top pane menus >Window >HLSL Code >Copy. And I got it to sample 1 MIP level by setting a Texture Sample Parameter node to MIP level 0, then finding it near the bottom of the output code.

I don't fully understand why you can't MIP texture samplers inside flow control (like IF or FOR statements), but apparently it's a fundamental feature of HLSL. You can find more info here http://stackoverflow.com/questions/15437413/directx-texture-sampling-function-samplecmplevelzero-returns-blank-intermittentl

Apologies if this doesn't apply to your post, but you're the one and only hit on google for >"ue4" HLSL for "cannot have gradient operations"

more ▼

answered Oct 27 '16 at 01:10 AM

avatar image

36 4 6 9

avatar image J. J. Franzen Oct 27 '16 at 09:59 PM

I would love to be able to verify but I haven't been able to get Unreal Editor to run under OS X for months now. Kinda given up on the whole thing.

(comments are locked)
10|2000 characters needed characters left
Viewable by all users
Your answer
toggle preview:

Up to 5 attachments (including images) can be used with a maximum of 5.2 MB each and 5.2 MB total.

Follow this question

Once you sign in you will be able to subscribe for any updates here

Answers to this question