#######################################################################
# Name:        README
# Purpose:     Describes dynamic mesh compression software module
# Peter van Beek, Tue Mar  2 15:18:11 1999
#######################################################################


OVERVIEW

This module contains software and data to perform dynamic 2D mesh
geometry and motion encoding and decoding. This tool implements the
Mesh Object compression tool as documented in the MPEG-4 Visual standard 
(ISO/IEC 14496-2) version 1. The module contains both encoder and
decoder, implemented in C++, which can be compiled and used as stand-
alone programs. The decoder can also be used integrated into the 2D/3D 
MPEG-4 Systems player, in which case it decodes data coming from a
BIFS-animation stream (ISO/IEC 14496-1). 

Most of what follows pertains to the stand-alone option and is of
interest only if you want to use the encoder/decoder stand-alone or
if you want to generate an animation stream for the Systems player.
If you just want to use the decoder integrated into the Systems player
and run some of the samples, you can skip most of what follows.

Two software programs are available:
- mesh_encode: encodes dynamic mesh geometry and motion into bitstream
- mesh_decode: decodes bitstream into dynamic mesh files 

There is no software included for:
- 2D dynamic mesh generation and tracking
- mesh-based video object rendering
Mesh generation/tracking precedes actual encoding and is non-normative.
Mesh-based rendering is handled in the context of scene-description,
composition and presentation. That is, the actual rendering will be
performed by the MPEG-4 Systems player and is non-normative as well.
However, a separate rendering utility is available for anyone who wants
to experiment with the rendering outside the MPEG-4 player.

The source files in this module are donated to ISO, subject to the 
copyright statement at the top of each file. The usual disclaimers
apply.

The software has been compiled and tested on both Unix (Solaris)
and Windows (MS Visual C++ 6.0) platforms.


FILES

The source and include files of the module are in the same directory.
The compiled object files and are put in a library in this directory.
The 'data' directory contains example dynamic mesh data. Some encoder
parameter files are included too. Compilation of the programs is
described below.


PROGRAMS 

1 - mesh_encode

In general, we have a set of mesh-files that each describe the
geometry of a 2D dynamic mesh at some point in time. These files were
obtained by tracking an initial mesh over the corresponding video
object (VO) but may also be generated synthetically in order to create
a certain graphics effect. This set of mesh-files is encoded by first
encoding the initial mesh geometry, and then encoding the node
displacements at each consecutive point in time. Here, the topology of
the mesh is not allowed to vary over time. The input mesh files all
have extension '.tri'.
All necessary parameters are passed using a parameter file. An example
parameter file for the Flag sequence is given in 'flag.par'. You
should first adapt this file to reflect the location of the input and
output files and directories you desire. 
To run the encoder program, just type:
	mesh_encode flag.par
and a bitstream will be generated that can be decoded by the mesh_decode
program. 

Note: a second encoder program can be created, mesh_enc_anim, which
generates a bitstream that can be decoded by the decoder in the Systems 
player.  This bitstream will be different than "stand-alone" bitstreams, 
due to a different wrapper syntax. This encoder is compiled from the
same source but with different compiler flags (see the Makefile).


2 - mesh_decode

The (stand-alone) mesh decoder decodes the above-mentioned bitstream
into the separate mesh-files discussed above. In case the original
mesh had Delaunay topology, the initial geometry is given by the node
point coordinates in the bitstream; the actual mesh is obtained by
Delaunay triangulation. In case the mesh had uniform topology, the
triangulation is obtained simply by enumeration of the triangles. For
the next time instances, this mesh is 'animated' by decoding the
motion vectors from the bitstream and applying them to the nodes.
To run the decoder program, just type:
	mesh_decode bitstream mesh_dec firstframe frameincr
where 'bitstream' is the bitstream file generated by 'mesh_encode' 
and 'mesh_dec' is the basename of the output mesh-files to be
generated. The actual filenames will be 'mesh_dec0.tri',
'mesh_dec1.tri', etc., depending on the actual frame numbers.
The parameter 'firstframe' determines the number given to the
first decoded mesh, the parameter 'frameinc' determines the 
increment value of the frame number going from one frame to the
next. 
The decoded output meshes should be identical to the original ones 
the 'data' directory (possibly upto a different arbitrary numbering 
of nodes and triangles, and upto a global displacement). 
One can supply the decoder with two more arguments <dx> <dy> which
specify a global displacement to be given to the initial mesh after
it is decoded. In a full MPEG-4 system, this is not necessary, since
such displacements are handled in the context of composition. When
experimenting without a full system, however, it is useful to be
able to place the mesh at its original (image-derived) location. 


MESH DATA

In this release, data for the Flag mesh sequence is provided, which
is a uniform topology mesh. There is also data for the Akiyo and
Bream sequences, both with object-based (Delaunay) topology. And 
there is another version of the Bream sequence with uniform topology.

The data directory is called 'data'. It contains the following:
flagunif*.tri    mesh data files for uniform 'Flag' sequence, 0-9
breamunif*.tri   mesh data files for uniform 'Bream' sequence, 60-110
bream*.tri       mesh data files for object-based 'Bream' sequence, 60-110
akiyo*.tri       mesh data files for object-based 'Akiyo' sequence, 0-75
cyclam*.tri      mesh data files for object-based 'Cyclam' sequence, 40-40

The '*.tri' files are simply ASCII files with the description of 
the mesh geometry at some point in time. 

Each '.tri' file contains a mesh object plane, and the file format 
used is simply as follows:

First, the number of nodes (=vertices) in the mesh is specified; then,
the number of triangles. Then follows a list of the nodes; then (after
an empty line), a list of the triangles.
For each node, the x and y coordinate is specified.
For each triangle, the indices of its three vertices in the node list
is given (these indices run from 0 to N-1 if there are N nodes). 

Note that (x,y) = (0,0) corresponds to the top-left corner of an
image, i.e., the center of the top-left pixel corresponds to the
location (x,y) = (0.5,0.5). Look for example at the mesh data in
flagunif0.tri, which corresponds to a uniform mesh overlaid on
the entire image (with width 320x240 pixels). 

Uniform meshes are assumed to always have their top-left point of
intra-coded mesh object planes in (0,0) and are coded as such. 
Object-based meshes may be generated having a top-left point
other than (0,0), but when coding them, the first intra-coded mesh is
used to setup a coordinate system such that its top-left point becomes
(0,0) in the new coordinate system. Subsequent intra-coded meshes may
have an offset with respect to this new origin. This way the compositor 
always knows where the (bounding box) of the mesh object is and where
it started in the sequence. If one would use the original coordinate
system, the compositor has to remember the actual offset of the object,
so that it could compute later how far the object has moved from its
original position. But in a broadcast scenario, one may tune in at
an arbitrary time, so the initial offset may not have been received.

Note that the current encoder allows encoding multiple sequences of 
IPPPP... frames; each such sequence is called a 'group of mesh object 
planes'. 
 

COMPILING

Windows:
--------
For compilation under Windows (WIN32), a MS Visual C++ 6.0 workspace 
and a project file is provided for each program. 

Unix:
-----
A Unix Makefile is provided for standalone compilation. You may have 
to adapt some definitions at the top of that file to reflect your 
system setup. Then, simply type 'make all'.

First, this will generate object libraries. Then, this will 
generate the binary executables 'mesh_encode', 'mesh_decode' 
and 'mesh_enc_anim'.


LITERATURE REFERENCES

C. Toklu, A.T. Erdem, M.I. Sezan and A.M. Tekalp, Tracking
motion and intensity variations using hierarchical 2-D mesh
modeling. Graphical Models and Image Processing, vol. 58,
no. 6, pp. 553-573, November 1996

C. Toklu, A.T. Erdem, M.I. Sezan and A.M. Tekalp, 2-D Mesh 
tracking for synthetic transfiguration. Proceedings Int. 
Conference on Image Processing, 1995

P.J.L. van Beek, A.M. Tekalp and A. Puri, 2-D Mesh geometry
and motion compression for efficient object-based video
representation. ICIP 97, Santa Barbara, CA, 1997

A.M. Tekalp, P. van Beek, C. Toklu and B. Gunsel, 2D Mesh-based 
visual object representation for interactive synthetic/natural 
digital video, Proceedings of the IEEE (Special issue on multimedia
signal processing), vol. 86, no. 6, June 1998, pp. 1029-1051


CONTACT 

Contact information of the original developers of these tools:
Prof. A.M. Tekalp
Department of Electrical Engineering
University of Rochester,
Rochester, NY 14627
USA
tekalp@ee.rochester.edu.

Peter van Beek
Sharp Laboratories of America
5750 N.W. Pacific Rim Blvd.
Camas, WA 98607
USA
pvanbeek@sharplabs.com

#######################################################################
