% -*- latex -*-
\section{Introduction}

Text parallelization and parallel corpora is important for natural
language processing, language studies, dictionary creation and can be
easily transformed on translation memories.

For this, we need to have parallel texts, but it is difficult to find
them. In this document we present a web based approach to retrieve
potential parallel texts from the world wide web, classify, and create
parallel corpora with them.

Figure \ref{dia:1} shows a general view of the process we will
present. Each down arrow number are respective to the following items:


\begin{enumerate}
  
\item To retrieve texts from the Internet we need to detect text
translations in diverse languages, and to make it correspond
interdependently. In section \ref{web:mining} we present two ways to
get these texts: using Resnik Parallel Strands, or using a new
approach, named Parallel Guessing.

\begin{figure}[t]
\begin{minipage}{0.45\textwidth}
$$web\vee directory$$
$$\Downarrow_1$$
$$(File^2)^\star$$
$$\Downarrow_2$$
$$(File^2)^\star$$
$$\Downarrow_3$$
$$((p^\star\times id)^2)^\star$$
$$\Downarrow_4$$
$$((p\times p)^\star\times id^2)^\star$$
\end{minipage}
\caption{General View of the Process\label{dia:1}}
\end{figure}
  
\item After the potential parallel texts have been fetched, a process
for validation is started. It will check the probability for these
texts to be parallel and keep only valid ones.  This process uses
auxiliary tools which will be discussed in sec.~\ref{validation}.
  
\item In the corpora building stage we split parallel texts into
sentences and convert them to a PML\footnote{Paragraph Markup Language
--- a XML document with only one element, the paragraph one}
file. 
  
\item Follows the alignment itself. In this task, we can generate
  translation memories (TMX\footnote{Translation Memory eXchange})
  back and forth.  

\end{enumerate}

To make these corpora useful, we decided to use the IMS
CWB to maintain them. In section \ref{cqp} we show how this
tool works, and how we can build CGI scripts to consult the parallel
corpus.

While this process can be used to produce the corpora itself, our
intent is not only the material preparation but also to construct a
set of tools to perform various tasks independently: align text files
which came from different sources than the web, or extract parallel
information from the web without making a real corpus with it.

On section \ref{eval} we present our evaluation for the algorithms and
techniques we use.

