Reconstructing editable prismatic CAD from rounded voxel models

Rather than attempting to produce arbitrary CAD modeling sequences, our approach focuses on the generations of B-Rep models which are built from pre-defined extrusion recipes, containing the axes along which profiles will be extruded, the order in which the extrusions are applied and whether each extrusion should be added or subtracted from the previous geometry. Most CAD software allows the designer to create sketches on existing planar faces of the solid being modeled, leading to some frequent patterns in the way extrusions are aligned and stacked. In this section we analyze the combinations of extrusions which designers employed when creating the models in the DeepCAD dataset (Wu et al., 2021) and find the most frequent patterns which are used as the basis for our extrusion recipes. After the deduplication procedure used in (Jayaraman et al., 2022), the dataset contains 127,267 files. Of these, just 358 contain two or more extrusions which are not orthogonal to one another, accounting for just 0.28% of the dataset. For this reason we focus only on the generation of solids built entirely from orthogonal extrusions. Without loss of generality, it is always possible to rotate orthogonal extrusion models such that the first extrusion will be created in the z direction. If a second extrusion direction exists, the model can then be rotated around the z axis so the x direction is aligned with the second extrusion direction. Transforming the models into this canonical pose allows 102,287 files (80.37%) of the dataset to be represented by the 18 combinations of extrusions shown in Appendix A.1  Figure 5. We see that single extrusions form the majority (56.7%) of the dataset. The next most frequent extrusion combination is a boss extruded from a sketch on the top face of the previous extrusion (see Figure 5b), which accounts for 5.24% of the models, followed by a cut which starts from the top face of the previous extrusion (see Figure 5c) which accounts for 4.47% of the dataset.

Many processes which produce signed distance function models apply some smoothing to the shape as a kind of regularization. To allow our method to work well with these rounded shapes, we augment the dataset as follows. For each model we can create a signed distance voxel representation of the geometry in the canonical orientation described in Section 3.1 and scaled into the unit cube. We then generate rounded augmentations of each model by adding a constant to the signed distance function to offset the shape inward, re-initializing the distance function at the zero levelset (Sethian, 1999) and reversing the process to offset outwards by the same amount. For each model in the dataset we add rounded augmentations with radii of 2.5%, 5.8%, 9.1%, 12.5% the size of the length of the bounding cube, corresponding to 1.6, 3.7, 5.8 and 8 voxels.

Constrained parametric sketches were obtained from the SketchGraphs dataset. As SketchGraphs contains collections of overlapping curves rather than closed profiles, the sketches were first processed to remove self-intersections. All intersecting curves were converted to construction geometry until the remaining geometry either formed closed loops or became disjoint. Disjoint sketches were discarded. An auto-constraining algorithm was then used to add as many constraints and driving parameters as possible without making the sketches over-constrained. This procedure was applied to 1,081,217 sketches from SketchGraphs and yielded 177,609 sketches which formed close loops and were suitable for downstream CAD operations. We then conducted an analysis of the sequences of lines and arcs present in profile loops in the DeepCAD dataset. The profile loops were clustered into groups which shared important geometric and topological properties as described in Appendix A.2. This analysis showed that 88% of profile loops in the DeepCAD dataset fall within the 50 largest groups. We then selected a small set of 49 constrained sketches which preserved the most common profile loop shapes over a wide range of parametric variations. Experiments showed that this small set of profile loops is able to approximate DeepCAD profiles surprisingly well, with an IoU of 93$\pm$11% as described in Section 6.1. Experiments with more complex sets of profiles showed these sometimes resulted in extraneous details in the generated solids. The parameters of the selected sketches were then varied to give rise to 1690 shapes (more detail in Appendix A.2). The closed loops of 2D curves in the sketches were converted to SVG and rendered as binary images with resolution of 128x128. The fast marching method was used to convert the data to signed distance functions with negative values inside the object.

An overview of the architecture for decoding the differentiable extrusion voxel models is shown in Figure 2. The input is a signed distance function, represented as a voxel grid with resolution 64x64x64. A standard voxel encoder as described in (Mescheder et al., 2019) can be used to convert this into a 128 length embedding vector $z$. This is passed to a series of decoder modules, each of which will attempt to interpret the shape in terms a predefined extrusion sequence recipe as described in Section 3.1.

For a decoder module with a sequence of $n$ extrusions, the voxel embedding, $z$, is split into extrusion embeddings $e_0,e_1,...,e_{n-1}$. This is done by a series of linear layers separated by ReLU non-linearities (see Appendix A.4). The extrusion embeddings can then be passed to a shared 2D deconvolution image decoder (see Appendix A.6) which will generate images with resolution 128x128, representing each profile, $P_0,P_1,...,P_{n-1}$. The decoded pixel values will be negative for pixels inside the profile and positive outside it.

In addition the locations of the start and end of the extrusions needs to be found. To achieve this we predict 1D arrays of envelope values which can be used to limit the extrusions in a differentiable way. If the $i$th extrusion does not share a common start or end plane with others in the model, we decode two arrays of length 64 to represent its limits. The start array, $S_i$ will contain positive values below the extrusions start plane and negative values above it, while the end array, $E_i$ will be negative below the end plane and positive above it. These arrays are decoded from the extrusion embeddings using a 1D deconvolution network as described in Appendix A.8. For some frequent cases where two extrusions share a common plane, a single start/end array can decoded and shared by both of them. This is explained in more detail in Appendix A.3. Taking the max of $S_i$ and $E_i$ defines an envelope which is negative only for the parts of the $i$th extrusion which we want to appear in the model.

With this information we can construct a 64x64x64 voxel grid representation of the model in a differentiable way. For each of the $n$ extrusions, the profile $P_i$ is first downsampled as described in Appendix A.7, and then duplicated along the appropriate dimension of a 3D tensor as defined by the hard coded recipe in the decoder module. The envelope is then applied by taking the max of each slice of the tensor with its corresponding value in the envelope array. Finally the extrusions are combined with the Boolean operations in the recipe, using the min function for union, max for intersection and multiplying by -1 to form the complement. This gives rise to a signed function, $\varphi$ of logits which, if passed through a sigmoid, would be the probability of a voxel being outside the model. Adopting this convention makes $\varphi$ negative inside the object, allowing the shape to be visualized directly from the logits using marching cubes. Note that $\varphi$ is not a distance function as the Eikonal equation is not satisfied as we move away from the zero level set.

The model is trained using two loss functions, a supervised loss which requires knowledge of the CAD construction sequence, and an unsupervised loss which does not require any construction sequence information. Both the supervised and unsupervised contributions to the loss are computed using a binary cross entropy ”with logits” loss which includes the sigmoid function allowing it to operate directly on the predicted logits

where $x$ is the tensor of predicted logits, $y$ is the binary target value, $\sigma \left(\right)$ is a sigmoid function and the mean is over all the elements of the tensor. For the unsupervised loss function, the target voxel values $\hat{T}$ are derived from the CAD data before any rounded augmentation is applied. $\hat{T}$ is 1 for voxels outside the object and 0 inside the object. The unsupervised loss for the differentiable voxel model is then

The supervised loss is computed using the ground truth 2D binary profile images, $\widehat{P_i}$ and 1D binary arrays, $\widehat{S_i}$ and $\widehat{E_i}$, corresponding to the start and end envelope arrays. Following the convention above, these tensors have values of 1 outside the object and 0 inside. We compute a binary cross entropy loss, averaged over the $n$ extrusions utilized by the decoder module as

The final loss is then computed as

During supervised training, the ground truth decoder module for each training example is known. At inference time this information is not available, so in order to make this selection we compare the input voxel model with the decoded differentiable extrusion model from each decoder module. The unsupervised binary cross entropy loss function, $L_{vox}$, can be evaluated for each of the available decoder modules and the module which results in the smallest loss is utilized. This procedure is also extended to allow regeneration of CAD models where the first extrusion is not in the z direction. The input voxel grid can be rotated 90 degrees around the x and y axes before the data is passed to the network. The differentiable extrusion model is then created for each decoder module with the target shape encoded in all three orientations. Then the best orientation and decoder module combination can be selected. Finally the regenerated CAD model is transformed back into its original pose.

At inference time the 2D profiles images and 1D envelope arrays can be used to create a parametric recipe suitable for use in a CAD system. The profile images are converted to CAD profiles using a search procedure in an embedding space defined by a 2D autoencoder. This is built from the CNN encoder and deconvolution decoder as described in Appendix A.5  and A.6. As in the 3D case, the input images are signed distance functions and the model is trained using the loss from Equation 3. Once trained, the 2D autoencoder can be used to generate embeddings for each variation of a collection of constrained parametric sketches. The 2D profile images created by the 3D decoder network can then be converted to binary images by selecting the pixels with values lower than zero. The resulting shapes can be separated into connected components, each of which will be converted into a single CAD profile. The shape used to search for the outer loop of each component can be found by filling any internal holes and the inner loops extracted as the difference between the original and filled shape. For each loop extracted in this way, we can zoom in on a square around its binary mask. As the 2D autoencoder is trained using signed distance functions as input, the zoomed images representing single loops need to be converted to signed distance functions using the fast marching method (Sethian, 1999). The search then finds the parametric sketch variation with the embedding closest, in terms of euclidean distance, to the embedding of the signed distance query image. Once the sketch has been retrieved, its parameters can be fitted to better approximate the query shape. In addition to the internal parameters of the parametric sketch, the translation in x and y and sketch scale factor can also be optimized. To rebuild the CAD model we also need the positions of the start and end planes for each extrusion. These are determined by finding the positions of zero crossings of the extrusion’s start and end arrays. If multiple zero crossings exist, the planes are chosen such that the length of the extrusion is maximized. A CAD kernel can then be used to generate a B-Rep extrusion of the retrieved profile. The Boolean operations from the decoder module’s recipe are then used to combine the extrusions to create the CAD model.

In this section we examine the performance of our 2D search retrieval and fitting technique at reproducing the shapes found in the profiles from 3,379 randomly selected models from the DeepCAD dataset. The 2D autoencoder was trained on 128,400 2D profiles generated by cutting axis aligned slices through 5,422 models in the ABC dataset. For each extrusion in the DeepCAD models, we create binary images from the union of the extruded profiles. The search and retrieval procedure described in Section 4.4 is then used to approximate the loops of the profiles using the 1,690 constrained parametric sketch variations created by the procedure described in Section 3.3. The IoU between the target profiles and the recovered sketches is 93$\pm$11% which is surprisingly high given the very small number of sketches over which the search is conducted. We also investigated using the binary mask as input to the search network rather than a signed distance function. This results in a substantially lowers the IoU to 84$\pm$21%. We speculate that using a signed distance function breaks the translation invariance in the CNN encoder architecture in a similar way to the solution suggested by (Liu et al., 2018). Figure 3 shows some randomly selected examples of target 2D shapes and the retrieved profiles. As 41.53% of DeepCAD profile loops are circles and 25.97% are rectangles, we see that the successful recovery of these simple shapes provides an excellent foundation for the search based approach.

Here we compare the performance of our reconstruction technique with DeepCAD. As our method can only generate the extrusion combinations for 80.37% of the models in the DeepCAD dataset, we use our own train/validation/test split of 70%/15%/15%. The official DeepCAD split provides only 5% of the models in the test set and does not account for duplicates. As target geometry, we utilize solids from the DeepCAD test set and from ABC. The ABC data is chosen such that can be built from extrusions, as described in Section 3.

Our network is trained on the DeepCAD dataset as described in Appendix A.9. We reconstruct CAD for 6,338 voxel models from DeepCAD and 7,876 models from ABC. For comparison with DeepCAD we use their pre-trained sequence encoder to generate sequence embeddings for our training set. We then train two encoders to replicate these embeddings. The original DeepCAD point cloud model is trained using point clouds extracted from triangulations of our voxels models, including the rounded augmentations. We also utilize our voxel encoder to consume the same voxel models as our network and produce the embeddings required for DeepCAD sequence generation. Both encoders are trained with MSE loss as in the official code from (Wu et al., 2021).

The results are shown in table 1. Our technique achieves the highest valid ratios of 83.84% on the DeepCAD dataset and 81.46% on ABC. DeepCAD has a tendency to create self-intersecting profile loops which result in invalid solids. In IoU, our method also outperforms DeepCAD. We observe that DeepCAD often creates plausible models with approximately the correct shape, but these do not fit as closely to the target shape as ours. We also record the IoU values for the differentiable voxel model produced by our approach. On the DeepCAD dataset this gives a value of $88\pm 12$% and for ABC $81\pm 13$%. This shows that between 7% and 9% of the mismatch between the target and final model results from CAD conversion. This is approximately consistent with the 2D search and retrieval results where we observe the parametric sketches can approximate the ground truth profiles with a mean IoU of 93%.

In FID, the DeepCAD model marginally outperforms ours on the DeepCAD dataset. We observe that DeepCAD tends to create data which is very close to the original dataset, with little adaptation to the target shape as shown quantitatively by the IoU metric. On the ABC dataset our method outperforms DeepCAD in FID, showing it is better able to generalize to different data distributions.

To show results qualitatively we randomly sample models from the DeepCAD experiments for display in Figure 4. Only 51% of the voxel models were successfully regenerated by all three methods. The DeepCAD dataset contains a large number of blocks and cylinders, so when we randomly sample examples from the successfully regenerated CAD, (see left of Figure 4), the results are dominated by these simple shapes. To show results of a wider variety of geometries we filter out blocks and cylinders, and then randomly sample the models in the middle of Figure 4. Our technique is able to generate valid solids in cases where the DeepCAD result was invalid. We show random samples from these on the right hand side of Figure 4. We show similar results using ABC as the target models in the appendix Figure 10. We observe that as the target geometry become more complex, our technique discovers simplified shapes which approximate it, while the DeepCAD results become random collections of simple extrusions.