On the fly assemblies in maxscript

posted in: Blog | 0

This is a guest post written by Klaas Nienhuis.

Part 2

This single piece of terrain actually consists of 9 separate meshes

In the previous part of the tutorial we read binary data from a file and built a mesh with pure maxscript. In this part we’re going to replace some of the operations with .NET and see if that improves the performance. We’re also going to slice the meshes to avoid having one big mesh. This should improve viewport performance a lot.

 

Check out the other parts in this tutorial

Part 1
Part 3

 

On the fly assembly

Within 3dsMax you can write with some kind of hybrid maxscript .NET code. While this expands the amount of possibilities of what you can do with scripting, it doesn’t necessarily improve speed. Especially when transferring values between .NET and maxscript stuff can actually slow down. As an alternative you can use an on-the-fly assembly. This works similar to a regular C# assembly you’d call from an external file. The on-the-fly assembly is created in memory only. The advantage is you can run almost pure C# code with all its speed benefits over maxscript. We’ll use that to get the data from the file. This particular piece of code is courtesy of denisT at cgtalk.

Code sample maxscript + .NET

One method has been exchanged since the last part. Data isn’t read anymore from a stream but by an on-the-fly assembly. Also a simple struct has been added and a method to create chunks from a larger grid.

 

(

   function fn_onTheFlyAssembly_readFileOps =

   (

   /*<FUNCTION>

   Description

   creates an on the fly assembly

   the method reads bytes from a binary file and returns integers. Also takes care of

   converting the data from big endian to little endian

   major parts of this code by denisT: http://forums.cgsociety.org/showpost.php?p=7838323&postcount=6

   Arguments

   Return

   <FUNCTION>*/

  

   --the C# code

   source  = ""

   source += "using System;n"

   source += "public class ReadFileOpsn"

   source += "{n"

   source += "    public Int16[] ReadFileShort(string file)n"

   source += "    {n"

   source += "    byte[] data = System.IO.File.ReadAllBytes(file);n"

   source += "    int len = Buffer.ByteLength(data);n"

   source += "    Int16[] result = new Int16[len / 2];n"

   source += "    for (int k = 0, i = 0; k < len; k += 2, i++)n"

   source += "    {n"

   source += "    result[i] = (Int16)(data[k] << 8 | data[k+1]);n"

   source += "    }n"

   source += "    return result;n"

   source += "    }n"

   source += "}n"



   --setting up the assembly in memory

   csharpProvider = dotnetobject "Microsoft.CSharp.CSharpCodeProvider"

   compilerParams = dotnetobject "System.CodeDom.Compiler.CompilerParameters"

   compilerParams.GenerateInMemory = on

   compilerResults = csharpProvider.CompileAssemblyFromSource compilerParams #(source)

   compilerResults.CompiledAssembly.CreateInstance "ReadFileOps"

   )

   

   struct str_chunk

   (

   pos = [0,0], --the position in samples

   segments = [100,100], --the amount of width and length segments for this chunk

   segmentSize = 90, --the size of a single segment. For srtm3 this is 3 arc seconds which is about 90 meters

   theDataIndices = #{} --indices for a data-array. These are the indices corresponding to this chunk. Storing the data itself will just take memory and is redundant

   )



   function fn_initDataGrid &outputmessage slices:5 gridSamples:1201  =

   (

   /*<FUNCTION>

   Description

   builds a grid of data structs based on a datagrid of a particular size

   each datastruct has a size and a position. they're created in such a way that they cover the input gridSamples

   the chunks tile across the datagrid north > south, west > east

   Arguments

   <value by reference> outputmessage: a message we're reporting to

   <integer> slices: the amount of width and length slices we want to slice the input grid into

   <integer> gridSamples: the amount of width and length samples the input grid has

   Return

   <array> an array of structs

   <FUNCTION>*/

  

   --calculate the sizes of the chunks based on the amount of slices you want to split the input grid into

   --adjacent chunks will share vertices

   local chunkSegments = [gridSamples as integer/slices as integer,gridSamples as integer/slices as integer]

   --if we're splitting into slices, add an extra row and column to allow for the overlap

   if slices > 1 do chunkSegments += [1,1]

   --make sure all samples are being used. add them to the chunks at the end of the row and column

   local lastChunkSegments = [chunkSegments.x + (mod (gridSamples-1) slices),chunkSegments.y + (mod (gridSamples-1) slices)]

   format "chunkSegments: %nLastChunkSize: %nAmount of chunks: %n" chunkSegments lastChunkSegments (slices^2) to:outputmessage



   --create and collect the datastructs

   local arrChunkData = #()

   for x = 1 to slices do for y = 1 to slices do

   (

   --build a chunk struct and determine its size and position in the datagrid

   local theData = str_chunk()

   theData.segments.x = if x == slices then lastChunkSegments.x else chunkSegments.x

   theData.segments.y = if y == slices then lastChunkSegments.y else chunkSegments.y

   theData.pos.x = (x-1)*(chunkSegments.x-1)

   theData.pos.y = (y-1)*(chunkSegments.y-1)

   format "tChunk %, position [%,%]n" ((x-1)*slices + y) theData.pos.x theData.pos.y to:outputmessage

  

   --create a bitarray which marks the needed data for this chunk from the acquired datagrid

   theData.theDataIndices[gridSamples^2] = false --initialize the bitarray

   for y = 1 to theData.segments.y do

   (

   local startByte = (theData.pos.y+y-1)*gridSamples + theData.pos.x

   for x = 1 to theData.segments.x do

   (

   theData.theDataIndices[startByte+x] = true

   )

   )

   append arrChunkData theData

   )

   arrChunkData

   )



   function fn_buildMesh theChunk =

   (

   /*<FUNCTION>

   Description

   builds a mesh object from rows and columns of heights. Intended to use with hgt files. this is data also known as srtm.

   uses a datastruct to determine what's being built

   Arguments

   <chunkData struct> theChunk: a datastruct, containing the info needed to create and translate the mesh

   <array> arrHeight: an array of heights as integer

   Return

   <mesh> the created mesh

   <FUNCTION>*/

  

   --build a planar mesh

   local theMesh = Editable_mesh wirecolor:(random (color 30 20 0) (color 30 30 10))

   setMesh theMesh

   width:((theChunk.segments.x-1)*theChunk.segmentSize)

   length:-((theChunk.segments.y-1)*theChunk.segmentSize) --a negative length puts the first vertex at the top left. This matches nicely with the data

   widthsegs:(theChunk.segments.x-1)

   lengthsegs:(theChunk.segments.y-1)

  

   --flip the normals because we set the length to a negative value

   addModifier theMesh (Normalmodifier flip:true)

   convertToMesh theMesh

  

   --place the mesh in the right position of the grid

   theMesh.position = [theChunk.pos.x*theChunk.segmentSize,-theChunk.pos.y*theChunk.segmentSize,0]

  

   update theMesh

   forceCompleteRedraw()

   theMesh

   )



   function fn_applyHeights theMesh arrHeight theIndices =

   (

   /*<FUNCTION>

   Description

   applies the heights to the vertices in the mesh

   Arguments

   <mesh object> theMesh: the mesh we're editing

   <array> arrHeight: an array of integers we'll use as heights

   <bitarray> theIndices: a bitarray which marks whichs heights need to be used

   Return

   <FUNCTION>*/

  

   local pos = theMesh.pos

   theMesh.pos = [0,0,0]

   --assigning heights to each vert. We're using the index stored in the indexarray of the chunk to access the correct height value

   local vertIdx = 1

   local meshvert = undefined

   local arrVert = for h in theIndices collect

   (

   meshvert = getVert theMesh vertIdx

   meshvert.z = arrHeight[h]

   vertIdx += 1

   meshvert

   )

   setMesh theMesh vertices:arrvert

   update theMesh

   theMesh.pos = pos

   )

   

   gc()

   local st = timeStamp()

   local mem = heapFree

   local msg = "" as stringstream

   local strFile = @"N:GitHubKML for 3dsMaxTestDataS23W068.hgtS23W068.hgt"

   

   local ReadFileOps = fn_onTheFlyAssembly_readFileOps()

   local arrInt = ReadFileOps.ReadFileShort strFile

   local arrChunkData = fn_initDataGrid &msg slices:3

   for chunk in arrChunkData do

   (

   local theMesh = fn_buildMesh chunk

   fn_applyHeights theMesh arrInt chunk.theDataIndices

   )
   

   format "Time: % ms, memory: %n" (timestamp()-st) (mem-heapfree)

   format "%" (msg as string)

)

 

Reading the data

The data is now read with the assembly and is a lot faster. The result is still an array of integers though, so the rest of the pipeline could stay the same.

Chunks

This terrain is split up into 9 chunks

 

Splitting incoming data into manageable chunks is a common task. I’ve built a small struct to represent a chunk. It’s as if we’re splitting a checkerboard into the separate checkers. Each checker refers to its own piece of the data and has an x/y coordinate. After defining the chunks, we create one mesh for each checker and position it on the board. We also associate the right parts of the data from the file to this chunk. Note that we don’t actually split the array with data into chunks, but just refer to the data with a bitarray. This is much faster and saves memory. After that it’s mostly the same as before. Don’t forget to place the checker in the right position on the board!

Evaluation

This version of the code runs in about 8 seconds which isn’t really better the first iteration. We do have better viewport performance because of the chunked up mesh. Also the data spikes are interpreted correctly now, they lie below the groundplane and don’t mess visually with the data.

Let’s see if we can speed it up by moving more calculations into C#. In the next part we’ll also create a real C# assembly in Visual Studio.

 

Check out the other parts in this tutorial

Part 1
Part 3

 

This is a guest post written by Klaas Nienhuis.
Original article: http://www.klaasnienhuis.nl/2014/08/fly-assemblies-maxscript/

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.