This is a guest post written by Klaas Nienhuis.
Part 2
In the previous part of the tutorial we read binary data from a file and built a mesh with pure maxscript. In this part we’re going to replace some of the operations with .NET and see if that improves the performance. We’re also going to slice the meshes to avoid having one big mesh. This should improve viewport performance a lot.
 |
 |
On the fly assembly
Within 3dsMax you can write with some kind of hybrid maxscript .NET code. While this expands the amount of possibilities of what you can do with scripting, it doesn’t necessarily improve speed. Especially when transferring values between .NET and maxscript stuff can actually slow down. As an alternative you can use an on-the-fly assembly. This works similar to a regular C# assembly you’d call from an external file. The on-the-fly assembly is created in memory only. The advantage is you can run almost pure C# code with all its speed benefits over maxscript. We’ll use that to get the data from the file. This particular piece of code is courtesy of denisT at cgtalk.
Code sample maxscript + .NET
One method has been exchanged since the last part. Data isn’t read anymore from a stream but by an on-the-fly assembly. Also a simple struct has been added and a method to create chunks from a larger grid.
(
function fn_onTheFlyAssembly_readFileOps =
(
/*<FUNCTION>
Description
creates an on the fly assembly
the method reads bytes from a binary file and returns integers. Also takes care of
converting the data from big endian to little endian
major parts of this code by denisT: http://forums.cgsociety.org/showpost.php?p=7838323&postcount=6
Arguments
Return
<FUNCTION>*/
--the C# code
source = ""
source += "using System;n"
source += "public class ReadFileOpsn"
source += "{n"
source += " public Int16[] ReadFileShort(string file)n"
source += " {n"
source += " byte[] data = System.IO.File.ReadAllBytes(file);n"
source += " int len = Buffer.ByteLength(data);n"
source += " Int16[] result = new Int16[len / 2];n"
source += " for (int k = 0, i = 0; k < len; k += 2, i++)n"
source += " {n"
source += " result[i] = (Int16)(data[k] << 8 | data[k+1]);n"
source += " }n"
source += " return result;n"
source += " }n"
source += "}n"
--setting up the assembly in memory
csharpProvider = dotnetobject "Microsoft.CSharp.CSharpCodeProvider"
compilerParams = dotnetobject "System.CodeDom.Compiler.CompilerParameters"
compilerParams.GenerateInMemory = on
compilerResults = csharpProvider.CompileAssemblyFromSource compilerParams #(source)
compilerResults.CompiledAssembly.CreateInstance "ReadFileOps"
)
struct str_chunk
(
pos = [0,0], --the position in samples
segments = [100,100], --the amount of width and length segments for this chunk
segmentSize = 90, --the size of a single segment. For srtm3 this is 3 arc seconds which is about 90 meters
theDataIndices = #{} --indices for a data-array. These are the indices corresponding to this chunk. Storing the data itself will just take memory and is redundant
)
function fn_initDataGrid &outputmessage slices:5 gridSamples:1201 =
(
/*<FUNCTION>
Description
builds a grid of data structs based on a datagrid of a particular size
each datastruct has a size and a position. they're created in such a way that they cover the input gridSamples
the chunks tile across the datagrid north > south, west > east
Arguments
<value by reference> outputmessage: a message we're reporting to
<integer> slices: the amount of width and length slices we want to slice the input grid into
<integer> gridSamples: the amount of width and length samples the input grid has
Return
<array> an array of structs
<FUNCTION>*/
--calculate the sizes of the chunks based on the amount of slices you want to split the input grid into
--adjacent chunks will share vertices
local chunkSegments = [gridSamples as integer/slices as integer,gridSamples as integer/slices as integer]
--if we're splitting into slices, add an extra row and column to allow for the overlap
if slices > 1 do chunkSegments += [1,1]
--make sure all samples are being used. add them to the chunks at the end of the row and column
local lastChunkSegments = [chunkSegments.x + (mod (gridSamples-1) slices),chunkSegments.y + (mod (gridSamples-1) slices)]
format "chunkSegments: %nLastChunkSize: %nAmount of chunks: %n" chunkSegments lastChunkSegments (slices^2) to:outputmessage
--create and collect the datastructs
local arrChunkData = #()
for x = 1 to slices do for y = 1 to slices do
(
--build a chunk struct and determine its size and position in the datagrid
local theData = str_chunk()
theData.segments.x = if x == slices then lastChunkSegments.x else chunkSegments.x
theData.segments.y = if y == slices then lastChunkSegments.y else chunkSegments.y
theData.pos.x = (x-1)*(chunkSegments.x-1)
theData.pos.y = (y-1)*(chunkSegments.y-1)
format "tChunk %, position [%,%]n" ((x-1)*slices + y) theData.pos.x theData.pos.y to:outputmessage
--create a bitarray which marks the needed data for this chunk from the acquired datagrid
theData.theDataIndices[gridSamples^2] = false --initialize the bitarray
for y = 1 to theData.segments.y do
(
local startByte = (theData.pos.y+y-1)*gridSamples + theData.pos.x
for x = 1 to theData.segments.x do
(
theData.theDataIndices[startByte+x] = true
)
)
append arrChunkData theData
)
arrChunkData
)
function fn_buildMesh theChunk =
(
/*<FUNCTION>
Description
builds a mesh object from rows and columns of heights. Intended to use with hgt files. this is data also known as srtm.
uses a datastruct to determine what's being built
Arguments
<chunkData struct> theChunk: a datastruct, containing the info needed to create and translate the mesh
<array> arrHeight: an array of heights as integer
Return
<mesh> the created mesh
<FUNCTION>*/
--build a planar mesh
local theMesh = Editable_mesh wirecolor:(random (color 30 20 0) (color 30 30 10))
setMesh theMesh
width:((theChunk.segments.x-1)*theChunk.segmentSize)
length:-((theChunk.segments.y-1)*theChunk.segmentSize) --a negative length puts the first vertex at the top left. This matches nicely with the data
widthsegs:(theChunk.segments.x-1)
lengthsegs:(theChunk.segments.y-1)
--flip the normals because we set the length to a negative value
addModifier theMesh (Normalmodifier flip:true)
convertToMesh theMesh
--place the mesh in the right position of the grid
theMesh.position = [theChunk.pos.x*theChunk.segmentSize,-theChunk.pos.y*theChunk.segmentSize,0]
update theMesh
forceCompleteRedraw()
theMesh
)
function fn_applyHeights theMesh arrHeight theIndices =
(
/*<FUNCTION>
Description
applies the heights to the vertices in the mesh
Arguments
<mesh object> theMesh: the mesh we're editing
<array> arrHeight: an array of integers we'll use as heights
<bitarray> theIndices: a bitarray which marks whichs heights need to be used
Return
<FUNCTION>*/
local pos = theMesh.pos
theMesh.pos = [0,0,0]
--assigning heights to each vert. We're using the index stored in the indexarray of the chunk to access the correct height value
local vertIdx = 1
local meshvert = undefined
local arrVert = for h in theIndices collect
(
meshvert = getVert theMesh vertIdx
meshvert.z = arrHeight[h]
vertIdx += 1
meshvert
)
setMesh theMesh vertices:arrvert
update theMesh
theMesh.pos = pos
)
gc()
local st = timeStamp()
local mem = heapFree
local msg = "" as stringstream
local strFile = @"N:GitHubKML for 3dsMaxTestDataS23W068.hgtS23W068.hgt"
local ReadFileOps = fn_onTheFlyAssembly_readFileOps()
local arrInt = ReadFileOps.ReadFileShort strFile
local arrChunkData = fn_initDataGrid &msg slices:3
for chunk in arrChunkData do
(
local theMesh = fn_buildMesh chunk
fn_applyHeights theMesh arrInt chunk.theDataIndices
)
format "Time: % ms, memory: %n" (timestamp()-st) (mem-heapfree)
format "%" (msg as string)
)
Reading the data
The data is now read with the assembly and is a lot faster. The result is still an array of integers though, so the rest of the pipeline could stay the same.
Chunks
Splitting incoming data into manageable chunks is a common task. I’ve built a small struct to represent a chunk. It’s as if we’re splitting a checkerboard into the separate checkers. Each checker refers to its own piece of the data and has an x/y coordinate. After defining the chunks, we create one mesh for each checker and position it on the board. We also associate the right parts of the data from the file to this chunk. Note that we don’t actually split the array with data into chunks, but just refer to the data with a bitarray. This is much faster and saves memory. After that it’s mostly the same as before. Don’t forget to place the checker in the right position on the board!
Evaluation
This version of the code runs in about 8 seconds which isn’t really better the first iteration. We do have better viewport performance because of the chunked up mesh. Also the data spikes are interpreted correctly now, they lie below the groundplane and don’t mess visually with the data.
Let’s see if we can speed it up by moving more calculations into C#. In the next part we’ll also create a real C# assembly in Visual Studio.
|
|
|
This is a guest post written by Klaas Nienhuis.
Original article: http://www.klaasnienhuis.nl/2014/08/fly-assemblies-maxscript/
Leave a Reply