#######################################################################
# Name:        README
# Purpose:     Describes mesh-based rendering utility software module
# Date:        Peter Van Beek, Fri Nov 13 17:57:05 1998
#######################################################################


OVERVIEW

This module contains software and data to perform dynamic 2D mesh-
based rendering of video objects. This module should be considered 
utility software, which may be combined with the module for dynamic 
mesh geometry and motion encoding and decoding. 
The latter (coding) module implements Mesh Object coding as documented 
in the MPEG-4 Visual standard (ISO/IEC 14496-2) version 1 and is a 
normative reference implementation. 
This rendering utility has proven useful during development of the 
MPEG-4 Visual standard. However, this (rendering utility) module is 
NOT a normative reference implementation. In the MPEG-4 context, mesh-
based rendering is handled in the context of scene-description,
composition and presentation of an MPEG-4 system. Actual rendering
will be performed by the MPEG-4 player. 
Another utility program is provided to compare the mesh data from
two different mesh sequences. This can be used to compare decoded
mesh object planes to reference mesh object planes from mesh
object sequences, so as to check decoders or bitstreams.

The rendering module is implemented in C++. 

Two software programs are available in this module:
- mesh_render: mesh-based 2D video object rendering
- mesh_cmp: compares files with mesh data

The source files in this module are donated to ISO, subject to the 
copyright statement at the top of each file. The usual disclaimers
apply.


FILE ORGANIZATION

There are 3 subdirectories: core / util / lib.  The 'core' directory
is the core / normative reference module (includes coding software) and
some mesh data. By using the makefile called 'Makefile.orig' in the 'core'
directory, a library with object code will be created in the 'lib' directory.

The 'util' directory contains the rendering utility software files and
some mesh and image data. Note that this software also depends on the
'core' module. By using the Makefile in 'util', another library in the
'lib' directory will be created.  Compilation and running the programs
is further described below.


PROGRAMS 

1 - mesh_render

Rendering a moving video-object (VO) over a number of frames,
from given texture and mesh-mappings.
A video object plane (VOP) refers to the texture and alpha
planes of a VO at one particular time instant.

A parameter file is used to communicate all necessary parameters to
the program. See e.g. 'renderflag.par' or 'renderbream.par'.

Types of rendering:
0 = self-mapping: mapping the first frame of the original
	sequence to all other frames of the same sequence;
1 = transfiguration: mapping a replacement object to all
	frames of the original sequence (replacing the original);
2 = augmentation: mapping an augmentation object over all
	frames of the original sequence (overlaid on original);
Types 0 and 2 are most useful for the data provided with this
module. 

In general, the rendering programs uses tracking data of a
moving video-object to generate (an approximation to that
video-object) from its first frame/VOP only. This is called
self-mapping. This first frame is called the reference frame in
this file.
The tracking data is in the form of a sequence of 2-D
triangular meshes, each describing the geometry of a 2-D mesh
at a certain point in time.
All the nodes in the mesh are present from the first to the last 
time instant (i.e. no nodes are ever added or deleted).

Optionally, the rendering can be augmented with another image/VOP,
such that it inherits the (deformable) motion of the original video
object. This is called augmentation and is performed by first using
another mapping from the augmentation VO to the reference frame/VOP of
the original VO. Then, again, the reference frame is rendered for the
other time instants.  We call transfiguration the act of replacing the
entire original VO with the rendering of another image/VOP.

As mentioned, all input parameters are specified using a parameter
file. A parameter file has been provided for the rendering of the
'flag' sequence, augmented with 'usa letters'. The name of the
parameter file is 'renderflag.par'. Another parameter file has been
provided called 'renderbream.par' for rendering the 'bream' sequence,
and for augmenting it with 'mpeg4' letters.

To run the program e.g. on flag, simply type 
	mesh_render renderflag.par
and the program will do the rest. Specifically, an output yuv sequence 
is generated containing the rendered sequence. You will need a display
program to see the result.


2 - mesh_cmp

Compares the mesh object planes in one sequence to the mesh object
planes in another sequence. A mesh object plane refers to a mesh
geometry from a dynamic mesh at a single time instant. The program
checks if for each node point in a mesh object plane of the second
sequence, there is a node with the same location in the corresponding
mesh object plane of the first sequence, and vice versa. The program
also checks if for each triangle in a mesh object plane of the second
sequence, there is a triangle that consists of the same node points
in the corresponding mesh object plane of the first sequence, and
vice versa.

The program arguments are:
	mesh_cmp <basename1> <basename2> <firstframe> <lastframe> <frame-inc> [-o <xo> <yo>] [-u] [-v] 
basename1:   basename of original mesh-files
basename2:   basename of reconstructed mesh-files
firstframe:  number of first frame of original sequence
lastframe:   number of last frame of original sequence
frame-inc:   frame increment of original sequence
-o xo,yo:    applies offset to second mesh
-u :         complains if mesh object plane vertices are not unique
-v :         verbose output

For instance, suppose we have encoded the mesh object stored in the
files bream60.tri, bream62.tri, bream64.tri, bream66.tri, bream68.tri
and bream70.tri into a bitstream. Then, we have decoded this bitstream 
into the files breamdec60.tri, breamdec62.tri, etc., breamdec70.tri.
To check whether encoding/decoding resulted in the same mesh
(up to a different numbering of nodes or global shift), use this command:
	mesh_cmp bream breamdec 60 70 2 -u
If there is a global shift, supply the x and y components of this
shift as the last two coordinates such that the two meshes are aligned.


MESH DATA

In this release, data for the 'flag' mesh sequence is provided, which
is a uniform topology mesh. 

The data directory is called 'data'. It contains the following.
The first set of files is necessary for any kind of rendering:
flagunif*.tri		mesh data files for 'flag' sequence, 0-9
flag.yuv         	raw yuv file with first texture plane of 'flag' sequence
flag.seg         	raw alpha plane file with first alpha plane of 'flag'

The second set of files is also necessary for augmentation of flag:
usa_letters.yuv     raw yuv file of 'letters' image, one frame
usa_letters.seg     raw alpha file of 'letters' image, one frame
usa_from.tri        mesh data file for 'letters' image
usa_to.tri          mesh data file for 'letters' image

The size of the frames of 'flag' are 480 lines by 640 pixels. 
The size of the 'letters' frame is 194 lines by 236 pixels.
Note that the program only uses the first frame of 'flag' and the 
mesh data files to render as sequence of frames. 
The motion of the 'flag' sequence is encoded in the 'flagunif*.tri'
mesh data files. The mapping of the augmentation image 'letters'
to the first frame of the original VO is encoded in the two files 
'usa_from.tri' and 'usa_to.tri'.

Also, data for the 'bream' mesh sequence is provided, which is an object-
based mesh with Delaunay topology. The data directory contains the following.
bream*.tri          mesh data files for 'bream' sequence, 60-110
bream.yuv           raw yuv file with first texture plane of 'bream' sequence
bream.seg           raw alpha plane file with first alpha plane of 'bream'

mpeg4.yuv           raw yuv file of 'mpeg4' image, one frame
mpeg4.seg           raw alpha plane file of 'mpeg4' image, one frame
mpeg4_from.tri      mesh data file for 'mpeg4 image
mpeg4_to.tri        mesh data file for 'mpeg4 image

The size of the frames of 'bream' are 288 lines by 352 pixels.
The size of the 'mpeg4' frame is 62 lines by 202 pixels. The motion
of the 'bream' sequence is encoded in the 'bream*.tri' mesh data
files. The mapping of the augmentation image 'mpeg4' to the first
frame of the original VO is encoded in the two files 'mpeg4_from.tri'
and 'mpeg4_to.tri'.


The '*.tri' files are simply ASCII files with the description of 
the mesh geometry at some point in time. 

Each ".tri" file contains a mesh object plane, and this file format, 
which allows triangular meshes, is used only for testing.

First, the number of nodes (vertices) in the mesh is specified; then,
the number of triangles. Then follows a list of the nodes; then (after
an empty line), a list of the triangles.  For each node, the x and y
coordinate is specified.  For each triangle, the indices of its 3
vertices in the node list is given.
Note that the indices of the nodes and triangles in the node list and 
triangle list start at 0 (not at 1). 


COMPILING
Windows:
--------
For compilation under Windows (WIN32), a MS Visual C++ 6.0 workspace 
and a project file is provided for each program. These are located
in subdirectories.

Unix:
-----
A Unix makefile is provided for standalone compilation, named
Makefile. You may have to adapt some definitions at the top 
of that file to reflect your system setup. 

Once you have have compiled the core module in the 'core' subdirectory,
then go to the 'util' directory to compile the rendering utility. 
You may have to adapt some definitions at the top of the Makefile 
to reflect your system setup. Then, simply type 'make all'.

First, this will generate/update an object libraries in the 'lib' 
directory. Then, this will generate the binary executable 'mesh_render'.



LITERATURE REFERENCES

C. Toklu, A.T. Erdem, M.I. Sezan and A.M. Tekalp, Tracking
motion and intensity variations using hierarchical 2-D mesh
modeling. Graphical Models and Image Processing, vol. 58,
no. 6, pp. 553-573, November 1996

C. Toklu, A.T. Erdem, M.I. Sezan and A.M. Tekalp, 2-D Mesh 
tracking for synthetic transfiguration. Proceedings Int. 
Conference on Image Processing, 1995


CONTACT 

Contact information of the original developers of this tool:
Prof. A.M. Tekalp, tekalp@ee.rochester.edu.
Department of Electrical Engineering
University of Rochester,
Rochester, NY 14627
USA

Peter van Beek, pvanbeek@sharplabs.com
Sharp Laboratories of America
5750 N.W. Pacific Rim Blvd.
Camas, WA 98607
USA

#######################################################################
