Strong Markov Random Field Texture Synthesis

My Home Page

Original Version

Fast Version

Fast Version on a Gaussian Pyramid with K Nearest Search

This algorithm has been recently published in IEEE Transactions on Pattern Analysis and Machine Intelligence 2004. The algorithm is based on the theory that an MRF with a much stronger conditional independence criteria, can be decomposed into its marginal distributions defined on its cliques. This stronger conditional independence criteria actually defines this strong-MRF on a triangulated graph. From graph theory [Judea Pearl "Probabilistic Reasoning in Intelligent Systems", 1988] and ANOVA log-linear construction [Bishop et al "Discrete Multivariate Analysis: Theory and Practice", 1975] it can be shown that this model can have its auto-model defined as combination of the marginal distributions defined on the pairwise cliques. I use this directly calculable auto-model to synthesis a variety of VisTex textures as presented on this webpage.

In the presented paper, I advocate using a Parzen density estimation of the pairwise marginal distributions with only a box kernel and a smoothing parameter equal to 0. This is possible using Ashikhmin's argument as presented in Synthesizing Natural Textures. Although this dramatically reduces the stochastic nature inherent in the model, it does allow to show that quite a variety of textures can be modelled with just second order statistics. This gives credence for using second order models for classification.

The model is not perfect. It is restrained by the assumptions made by the theory on the texture, and in the current implementation the Parzen density estimation of the pairwise marginal distributions does not take into account the stochastic nature of texture. The Parzen density estimation of the pairwise marginal distributions needs to be estimated with respect to entropy as in Zhu, Wu and Mumford's minimax entropy model ["FRAME: filters, random fields, and minimax entropy towards a unified theory for texture modeling", Proceedings 1996 IEEE Computer Society Conference on Computer Vision and Pattern Recognition].

In the synthesis results presented here, I have experimented with different Parzen smoothing (ie window) parameters. For an input parameter of window, I find a window size that equals the average distance to the first k nearest neighbours, where

k = ImageSize^(4/7)*window/100

This value of k is from on Silverman ["Density estimation for statistics and data analysis", 1986] who says on page 99, that; "In the case of the generalized nearest neighour method .... the ideal k .... would be proportional to ImageSize^(4/(d+4)." As the data is of RGB pixels, I have used the value of k as given above.

The synthesis algorithm employed is based on my earlier algorithm for nonparametric MRF texture synthesis. It is a multiscale algorithm, meaning that ImageSize is dependent on scale. Calculating k at each scale gives a small window size at low resolution with increasing window size at each higher resolution. This fits neatly in with De Bonet's work ["Multiresolution sampling procedure for analysis and synthesis of texture images", SIGGRAPH: Computer Graphics, 1997] who employed a threshold that allowed the "variability" to increase with each higher resolution.

Source Code

Download Source Code Copyright (C) 2002, by Rupert Paget, all rights reserved.

The source code requires ImageMagick to be preinstalled. To compile the code, run gmake in the parent directory. There are two makefiles, one in parent directory, and in the src directory. If the code does not compile, both makefiles will need to be edited. I have written one for a sun (default) and one for an sgi machine, but I give no guarantees that they will work.

The program is executed with the following command

nonparaMRF_strong [-l levels] [-n neighOrder] [-s] [-t treeMax] [-i iterations] [-w window] [-c] [-x cols] [-y rows] [-m maskfile] [-h] texturefile

where

levels =
number of gray levels the image will be compacted to. Reducing the number from 255, will probably not improve the synthesis, and will most likely increase the run time.
neighOrder =
number to signify neighbourhood size. See neighbourhoods for schematics of the neighbourhoods used in the texture synthesis algorithm.
s =
if set, the neighbourhood becomes a square neighbourhood.
treeMax =
maximum quadtree height. The algorithm will not allow the quadtree height to be set to a value that will allow either the input image or output image to be reduced to less than a 2x2 pixel image.
iterations =
number of iterations to be performed over the whole image.
window =
Parzen smoothing window parameter. The actual window size is calculated as given above.
c =
if set, the output image is defined to be toroidal (ie cyclic) where each border matches with its opposite border.
cols =
number of columns in the output image. If undefined, number columns in the output image will equal number columns in the input image.
rows =
number of rows in the output image. If undefined, number rows in the output image will equal number rows in the input image.
maskfile =
is a mask image that must be the same size as the input texture image. Only where the mask image is non zero is the input image used.
h =
if set, will mean that just usage and default values will be displayed. This will also occur if no input values are given.
texturefile =
is the input texture image that is to be synthesised.

Default values are : levels = 256, neighOrder = 1, square = false, treeMax = 6, iterations = 1, window = 0, cyclic = false

Schematic of the neighbourhoods used in the texture synthesis algorithmSchematic of the neighbourhoods used in the texture synthesis algorithm

Synthesis run times for various input parameters

These times were compiled on a distributed system of mainly SunBlades 100: 500 MHz, 256 MBytes RAM. The input textures were 128x128 pixel images, and the output synthesised textures were 256x256 pixel images. The run times are basically dependent on output image size and the type of input texture. This is due to the search algorithm used in the code. If the input texture is fairly uniform in colour, then the search algorithm will be more of an exhaustive search, and therefore more dependent on input image size. Time stamps are in "Days Hours:Minutes:Seconds".

NeighbourhoodIterationsWindow# of TestsFastest Run TimeSlowest Run TimeMean Run TimeStdev Run Time
1 -s 2 01650 00:20:360 02:23:260 00:49:510 00:52:52
1 -s 2 11650 00:20:170 02:15:570 00:52:040 00:56:14
1 -s 2 21650 00:20:400 02:05:290 00:54:300 00:57:33
1 -s 2 51650 00:21:030 02:21:000 01:00:270 01:03:45
1 -s 2101650 00:22:190 03:23:510 01:13:160 01:18:38
1 -s 2201650 00:42:510 03:11:260 01:38:570 01:44:30
2 -s 2 01650 00:42:040 03:33:010 01:47:500 01:53:34
2 -s 2 11650 00:43:510 03:46:010 01:51:060 01:57:23
2 -s 2 21650 00:44:220 03:56:170 01:54:030 02:00:32
2 -s 2 51650 00:42:260 06:25:510 02:08:100 02:16:58
2 -s 2101650 00:45:590 05:52:280 02:24:340 02:32:38
2 -s 2201650 01:13:370 06:09:120 03:08:430 03:19:42
3 -s 2 01650 01:15:270 09:23:110 03:21:460 03:37:40
3 -s 2 11650 01:15:120 11:34:160 03:26:130 03:44:36
3 -s 2 21650 01:15:460 10:24:210 03:34:170 03:48:51
3 -s 2 51650 01:15:200 10:57:200 03:50:020 04:06:04
3 -s 2101650 01:19:030 13:59:070 04:24:190 04:28:46
3 -s 2201650 02:00:270 18:26:090 05:12:290 05:21:11
4 -s 2 01650 01:54:340 12:39:000 05:14:560 05:35:18
4 -s 2 11650 02:02:480 11:51:390 05:18:490 05:41:03
4 -s 2 21650 01:53:470 15:37:250 05:33:140 05:36:28
4 -s 2 51650 02:05:520 14:46:070 06:00:180 05:54:53
4 -s 2101650 02:10:480 12:57:240 06:38:430 06:52:57
4 -s 2201650 02:50:300 18:18:050 07:38:300 07:22:53

Limitations

Additional Information

For More Information

Email Rupert Paget at: dr@rupertpaget.com

Valid HTML 4.01!

This site was proofread by
English Editing Experts.