Quantifying Communication in Synchronized Languages Zhe Dang , Thomas R. Fischer

Quantifying Communication in Synchronized
Languages
Zhe Dang1,2 , Thomas R. Fischer2 , William J. Hutton, III2 , Oscar H. Ibarra3 ,
and Qin Li1
1
School of Computer Science and Technology,
Anhui University of Technology, Ma’anshan, China
2
School of Electrical Engineering and Computer Science,
Washington State University, Pullman, WA 99164, USA
3
Department of Computer Science
University of California, Santa Barbara, CA 93106, USA
Abstract. A mutual information rate is proposed to quantitatively evaluate inter-process synchronized communication. For finite-state processes
with implicit communication that can be described by a regular counting
language, it is shown that the mutual information rate is effectively computable. When the synchronization always happens between the same
two symbols at the same time (or with a fixed delay), the mutual information rate is computable. There are other variations of the problem
for which the mutual information rate is shown computable. Finally, it
is shown that some cases exist where the mutual information rate is not
computable.
1
Introduction
Computer systems often run a set of processes communicating with a provided
interface for processes to talk with each other. In this way, the system may
implement highly complex functions; e.g., a concurrent system. Obviously, communications between two processes contribute significantly to the complexity of
the whole system. Therefore, it is desirable to find a way to measure the quantity of the communications between two processes. Since such communications
are often nondeterministic, the quantity would also be part of an indication of
hardness of testability when an analyst tests the system.
However, defining such a quantity is not trivial. Static analysis [3, 19, 23]
would not always work because the quantity we are looking for is a dynamic
indicator on how “tightly” two communicating processes are working together.
Another idea would be to use the classic theory of communication complexity
[12, 21, 17]. However, this theory aims at studying the minimal communication
bit-rate needed for a given communication task instead of analyzing the actual
amount of communication involved between given processes [28].
In this paper, we provide a metric to quantify the amount of communication
between two communicating processes where the communication mechanism is
synchronous. Our ideas are as follows. When a process runs, it demonstrates
a behavior, which is a sequence (or a word) of events. In this way, two processes, when run synchronously, demonstrate two parallel aligned sequences. In
automata theory, the parallel sequences can be thought of a word of two tracks
called a synchronized word. The set of all such synchronized words that are the
actual runs of the two communication processes forms a synchronized language,
written as L12 . The metric we study is defined as a type of dependency between
two tracks (each called a string, and both strings are of the same length) in a
synchronized word of the language.
Problem Given two strings with the same length, determine the communication
between them.
One can use mutual information to seek a solution. Suppose we have two strings,
X and Y . The average mutual information between them is
I(X; Y ) =
X
PX,Y (x, y) log2
x,y
PX,Y (x, y)
.
PX (x)PY (y)
(1)
Manipulating, this becomes.
I(X; Y ) = H(X) + H(Y ) − H(X, Y ).
(2)
The generalization to strings of length n (i.e., X = (X1 , · · · , Xn ) is
I(X; Y) = H(X) + H(Y) − H(X, Y).
(3)
In order to calculate these, we need the joint probability PX,Y (x, y) and the
marginal probabilities PX (x) and PY (y) for the first case, and the joint probability PX,Y (x, y) and joint marginal probabilities PX (x) and PY (y) for the
second case. If the string length is significantly larger than one, it can be very
difficult to obtain the required probabilities in practice.
Fortunately, there is a definition of the information that does not involve the
probabilities. We now consider information rate (i.e. the average information
quantity, measured in bits, per symbol in a word) instead of information. This
definition was proposed by Shannon [22] and later Chomsky and Miller [6], that
we have evaluated through experiments using C programs [27, 18, 10, 7, 8]. For a
number n, we use Sn (L) to denote the number of words in a language L whose
length is n. The information rate λL of L is defined as λL = lim λn,L , where
λn,L = log Snn (L) . When the limit does not exist, we take the upper limit, which
always exists for a finite alphabet. Throughout this paper, the logarithm is to
base 2.
The intuition behind Shannon’s definition is as follows. λn,L specifies the average number of bits needed per symbol, i.e., bit rate, if one losslessly compresses
a word of length n in L, while the information rate λL is simply the asymptotic
bit rate. In other words, λL is the average amount of information per symbol
contained in a word in L.
2
Based on this definition, the metric we are looking for is the mutual information rate, modified from (3):
ηL12 = λL1 + λL2 − λL12 ,
(4)
where L1 and L2 are the projections of the aforementioned synchronized language L12 to its first and second tracks, respectively.
In the paper, we show cases when computing the mutual information rate is
effective. These cases assume that the two processes are finite-state but the implicit communication mechanism between the two makes the resulting synchronized language L12 rather complex; e.g., a regular counting language (a regular
language constrained by a Presburger formula on counts of individual symbols
in a word). We show that when the synchronization always happens between two
symbols that are the same and that are at the same time, the mutual information
is computable. The proof is quite complex, which involves combinatorial analysis
of the synchronized words in L12 and properties from reversal-bounded counter
machines [13]. Later, we also show that this result can be further generalized
to cases when the two symbols are not necessarily synchronized at the same
time (with a fixed delay). However, the case of arbitrary delays is open. In some
cases, the mutual information rate is not computable. Cases include scenarios
when the two processes are of infinite-state; e.g., restricted counter automata
and pushdown automata. We present some of these cases as well.
2
Quantifying communication with information rate
Let Σ be an alphabet and consider two languages L1 and L2 on the alphabet Σ.
For the purpose of this paper, a word on the alphabet represents an observable
behavior of a process, which is a sequence of its observable events. Such an event
can be, for instance, a state transition when the states are observable. Suppose
that Pi (i = 1, 2) is a process One can think the two processes as two nodes
on a network, two concurrent programs running on a processor, two persons
monitored by a surveillance cameras in a building, or simply two people dancing
on a stage. When the two processes are observed together, a joint behavior is
obtained. To ease our presentation, we assume that, whenever an event (say a)
is observed in one process, an event (say b) is also observed in another process.
Hence, the two processes, intuitively, run at the same pace by observation. Actually, this assumption is made without loss of generality. This is because, one
can always introduce a new “idle” event into Σ. At the time when a is observed
in one process and, at this time, if no event is observed in another, we treat the
“no event” as the idle event. In this way, the two processes still run at the same
pace by observation.
With this formulation, a joint behavior is a synchronized word α on alphabet
Σ k . Without loss of generality, we take k = 2 (i.e., two processes). It is trivial
to generalize all the results in the paper to an arbitrary k. In the two process
case, a synchronized word, α, is of the form
(a11 , a21 ) · · · (a1n , a2n )
3
(5)
for some n and some words w1 = a11 · · · a1n and w2 = a21 · · · a2n in Σ ∗ . For notational convenience, we often write
α = [w1 , w2 ]
while, implicitly, we assume that the two projections w1 and w2 share the same
length. In the sequence, w1 is called the first coordinate of α while w2 is the
second coordinate of α.
The synchronized word α can be thought of as being woven from its first
coordinate w1 and second coordinate w2 . When one thinks of a joint run of
two synchronized processes as a synchronized word, a restriction may be placed
on the possible α so that not every pair of w1 and w2 can be woven into an
actual joint run. For instance, under the scenario that an event a in process
P1 must be synchronized with an event a˙ in process P2 , the synchronized word
[aba, a˙ a˙ a]
˙ cannot be a joint run. The exact form of the restriction can be an
explicit definition of the communication mechanism used between the processes
or an implicit constraint on the set L12 of all the synchronized words α observed
as joint runs. We take the latter approach. That is, in other words, we are
given a synchronization predicate R(w1 , w2 ) to define the L12 : for all w1 and w2 ,
[w1 , w2 ] ∈ L12 iff R(w1 , w2 ). Implicitly, R(w1 , w2 ) assumes that the lengths of
w1 and w2 are the same. We use L1 to denote the projection of L12 onto the
first coordinate; i.e., L1 = {w1 : [w1 , w2 ] ∈ L12 }. L2 is defined similarly.
The information rate λL12 , defined in Section 1, can be thought of the entropy
of a pair of random variable (X, Y ) with certain joint distribution p(X, Y ). Then,
the information rate λL1 (resp. λL2 ) is naturally the entropy of random variable
X (resp. Y ) with marginal distribution p(X) (resp. p(Y )) induced from the joint
distribution p(X, Y ). In this sense, the mutual information rate between L1 and
L2 with respect to the synchronized language L12 is defined as
ηL12 = λL1 + λL2 − λL12 ,
(6)
which can be intuitively seen from the following classic Venn diagram of Shannon
entropy [4].
The mutual information rate ηL12 can be used in quantifying the amount
of communication in the synchronized language L12 . In other words, suppose
that P1 (resp. P2 ) is a process whose observable event sequences are exactly
those in L1 (resp. L2 ). When P1 and P2 run synchronously, a synchronization
mechanism (which is a form of communication) may be placed in the processes
so that the resulting joint run is guaranteed to be in L12 . Now, when there is no
communication between the two, the two processes run independently and hence
L12 is simply the “Cartesian product” [L1 , L2 ] = {[w1 , w2 ] : w1 ∈ L1 , w2 ∈ L2 }
of L1 and L2 . In this case, one can verify that ηL12 = 0. On the other hand,
when P1 and P2 must synchronize on every event; e.g., L12 = {[w1 , w2 ] : w1 =
w2 , w1 ∈ L1 , w2 ∈ L2 } and L1 = L2 , one can verify that ηL12 = λL1 = λL2 .
That is, the synchronization mechanism makes sure that P1 and P2 share the
complete information. Intuitively, ηL12 characterize the average number of bits
transmitted between the two processes per event observed, asymptotically.
4
λL1 + λL2 − λL12
λL12 − λL2
λL12 − λL1
|
{z
λL12
}
Fig. 1. The left circle represents λL1 while the right one represents λL2 . The middle
overlapped area is λL1 + λL2 − λL12 , which represents the mutual information rate.
The diagram replicates the traditional Venn diagram of Shannon’s mutual information,
joint information and conditional information.
3
Computing mutual information rate in synchronized
languages
Recall that, in order to compute the mutual information rate in a synchronized
language L12 , it suffices for us to calculate the information rates λL1 , λL2 and
λL12 , according to (4). That is, we need algorithms to compute the information
rate of a language drawn from a class. The following is a fundamental result.
Theorem 1. The information rate of a regular language is computable [6].
The case when L is non-regular (e.g., L is the external behavior set of a software
system containing (unbounded) integer variables like counters and clocks) is
more interesting, considering the fact that a complex software system nowadays
is almost always of infinite state and the notion of information rate has been used
in software testing [7, 25]. However, in such a case, computing the information
rate is difficult (sometimes even not computable [15]) in general. Existing results
(such as unambiguous context-free languages [16], Lukasiewicz-languages [24],
and regular timed languages [1]) are limited and mostly rely on Mandelbrot
generating functions and the theory of complex/real functions, which are also
difficult to generalize. A recent important result [5], using a convex optimization
technique, will be used in the paper. First, some definitions are needed.
A counter is a non-negative integer variable that can be incremented by 1,
decremented by 1, or remain unchanged. Additionally, a counter can be tested
for equality with 0. Let k be a non-negative integer. A non-deterministic kcounter machine (NCM) is a one-way non-deterministic finite automaton, with
input alphabet Σ, augmented by k counters. For a non-negative integer r, we
use NCM(k,r) to denote the class of k-counter machines where each counter is
5
r-reversal-bounded; i.e., it makes at most r alternations between non-decreasing
and non-increasing modes in any computation; e.g., the following counter value
sequence
0012233210011
is of 2-reversal, where the reversals are underlined. For convenience, we sometimes refer to a machine M in the class as an NCM(k,r). In particular, when
k and r are explicitly given, we call M a reversal-bounded NCM. When M is
deterministic, we use ‘D’ in place of ‘N’ (e.g., DCM). As usual, L(M ) denotes
the language that M accepts.
Reversal-bounded NCMs have been extensively studied since their introduction in 1978 [13]; many generalizations are identified; e.g., multiple tapes,
two-way tapes, stacks, etc. In particular, reversal-bounded NCMs have found
applications in areas like Alur and Dill’s [2] time-automata [9, 11], Paun’s [20]
membrane computing systems [14], and Diophantine equations [26].
In a recent paper [5], we have shown the following result.
Theorem 2. The information rate of the language accepted by a reversal-bounded
DCM is computable.
Notice that, in the theorem, the case for a reversal-bounded NCM is open.
We now recall a number of definitions that will be useful later. Let N be
the set of non-negative integers and k be a positive number. A subset S of N k
is a linear set if there are vectors v0 , v1 , · · · , vt , for some t, in N k such that
S = {v|v = v0 + b1 v1 + · · · + bt vt , bi ∈ N }. S is a semi-linear set if it is a finite
union of linear sets. Let Σ = {a1 , · · · , ak } be an alphabet. For each word α ∈ Σ ∗ ,
define the Parikh map of α to be the vector #(α) = (#a1 (α), · · · , #ak (α)), where
each symbol count #ai (α) denotes the number of symbol ai ’s in α. For a language
L ⊆ Σ ∗ , the Parikh map of L is the set #(L) = {#(α) : α ∈ L}. The language
L is semi-linear if #(L) is a semi-linear set. The following theorem is a classic
result needed in this paper:
Theorem 3. Let M be a reversal-bounded NCM. Then #(L(M )) is a semilinear set effectively computable from M [13].
Let Y be a finite set of integer
P variables. An atomic Presburger formula on
Y is either a linear constraint y∈Y ay y < b, or a mod constraint x ≡d c, where
ay , b, c and d are integers with 0 ≤ c < d. A Presburger formula can always
be constructed from atomic Presburger formulas using ¬ and ∧. Presburger
formulas are closed under quantification. Let S be a set of k-tuples in N k . S is
Presburger definable if there is a Presburger formula P (y1 , · · · , yk ) such that the
set of non-negative integer solutions is exactly S. It is well-known that S is a
semi-linear set iff S is Presburger definable.
In the rest of the section, we focus on the case when the synchronization
predicate R in defining the L12 is given, how we are going to compute the
mutual information rate ηL12 ? In practice, such a predicate may be unknown
(e.g., a black box implementation of a concurrent system). We will investigate
6
how to estimate the mutual information rate ηL12 for this case in a forthcoming
paper.
When R is regular (i.e., the synchronized language L12 defined by R is a
regular language), both L1 and L2 are also clearly regular languages. Then from
Theorem 1 and the definition in (4), we have,
Theorem 4. The mutual information rate in a regular synchronized language
is computable.
The theorem implies that, for finite state processes that run synchronously, the
amount of communication between the two can be computed effectively.
A simple example would be two processes using a common resource as a
covert communication channel. Imagine one process, CW , is a covert writer, and
another process, CR , is a covert reader. In order to implement error-checking to
ensure 100% accuracy when transmitting a message, CW will not begin writing
symbols until CR acknowledges that it is ready to receive a symbol by synchronizing with CW . When CR has received a symbol from CW , CR moves its tape
head left and overwrites the previous symbol on the tape with symbol it just
received from CW . CW is idle until the symbols match. This enables CW to wait
until CR has acknowledged the sent symbol before sending the next symbol. CW
sends an end-of-file symbol at the end of the message.
When R is not regular, computing the mutual information rate of L12 is
more difficult and, in many cases, is not even effective. This is because, in doing
so, the projected languages L1 and L2 are likely accepted by non-deterministic
automata and hence, as we have mentioned earlier, their information rates in (4)
are known to be computable for only a few cases. We now investigate the case
for a non-regular R where counts of symbols are involved. More precisely, we
employ a vector of variables #(a,b) , a, b ∈ Σ and consider a Presburger formula
P over these variables. For a synchronized word α in (5), we use #(α) to indicate
the vector of all Parikh maps #(a,b) (α) (i.e., the number of pairs (a, b) appearing
in α), a, b ∈ Σ. A synchronized word α in (5) satisfies P if P (#(α)) holds.
We say that R is a regular counting predicate if it is specified by [Q1 , Q2 ] ∧ P
where Q1 and Q2 are regular predicates and P is a Presburger formula. In
consequence, L12 that is defined by R is a regular counting language; i.e., L12 is
the intersection of
ˆ 1 = {w1 : Q1 (w1 )} and
– the Cartesian product of the regular languages Q
ˆ 2 = {w2 : Q2 (w2 )}, and
Q
– the counting language Pˆ = {α : P (#(α))}.
In other words, L12 is defined as
ˆ1, Q
ˆ 2 ] ∩ Pˆ .
[Q
By definition, a synchronized word [w1 , w2 ] is in L12 iff
ˆ 1 and w2 ∈ Q
ˆ 2 , and
– (a). w1 ∈ Q
– (b). [w1 , w2 ] ∈ Pˆ .
7
Observe that condition (a) above specifies a condition independently applied on
w1 and w2 . However, for condition (b), one can imagine an explicit communication mechanism between w1 and w2 is placed to enforce the condition P . We now
further specify an explicit (regular) synchronization mechanism Syn between the
two. As a result, the L12 is now a synchronized regular counting language and
defined by
ˆ1, Q
ˆ 2 ] ∩ Pˆ ∩ Syn.
ˆ
[Q
(7)
We now investigate the cases when the mutual information rates for synchronized regular counting languages are computable.
Syn is used to specify an explicit synchronization pattern between two state
sequences. A common pattern is to force certain state pairs to move together;
e.g., dancer A jumps iff dancer B jumps. Formally, we are given a subset of synchronized symbols Π ⊆ Σ and define Σ − = Σ − Π to be the set of independent
symbols. The first pattern we consider is depicted as in Figure 2.
{
t
P−
}∗
z
z
}|
|
{z
α1
α2
k
{
a?
z }| {
b?
_
_
} a | {z } b
f
z
}|
{
|
{z
=
}
{
P−
a?
/

/
a_
}∗
Fig. 2. Scenario of 0-delay Syn where a, b ∈ Π.
ˆ
In other words, Syn
defines a regular language on alphabet (Σ − × Σ − ) ∪
{(a, a) : a ∈ Π}. We call such a Syn 0-delay.
Consider this formal description of the previously mentioned covert writer.
CW is (Q, Σ, δ, q0 , F ) where
1. Q ={q0 , q1 , q2 , q3 , q4 , q5 }
2. Σ ={any printable character, e.g. 0x20 to 0x5F in ASCII}
3. δ is given as
8
ENQ Σ EOF δ
q0 {q1 } ∅
∅
∅
∅ {q4 } {q2 }
q1 ∅
q2 ∅
∅
∅ {q3 }
∅
q3 ∅ {q1 } ∅
q4 ∅
∅
∅ {q5 }
q5 ∅
∅
∅
∅
4. q0 is the start state, and
5. F ={q5 }
Synchronization between CW and CR occurs at two points. First, CW remains in q0 until it receives ”ENQ” from CR . Then CW transitions into a work
loop that uses states q1 , q2 , and q3 to transmit a character to CR . The second
synchronization point occurs when CW is in state q3 . CW is blocked until it
receives the character it sent from CR . When CW receives this character, CW
becomes unblocked and can transition back to state q1 and repeat the work
loop for the next character to send until CW reaches an end-of-file marker and
transitions to state q4 and finally q5 , which is an accepting state.
Theorem 5. The mutual information rate in a synchronized regular counting
language, specified in (7) with a 0-delay Syn, is computable.
The second pattern of Syn is a slight generalization of the first, namely ddelay Syn, which is depicted in Figure 3. It says that any two corresponding
synchronization symbols are separated away by exactly d symbols. Clearly, such
a Syn is also regular.
α1
α2
a. N
a. N
b. N
n
n
_o
d
a
/
/
_o
d
b
/
n
_o
d
a
/
/
Fig. 3. Scenario of d-delay Syn
It is straightforward to generalize the proof of Theorem 5 to d-delay Syn.
Theorem 6. The mutual information rate in a synchronized regular counting
language, specified in (7) with a d-delay Syn, is computable, for any given d.
Recall that Syn is intended to describe a “regular” communication mechaˆ
nism between two processes; i.e., the corresponding Syn
defines a regular language (of synchronized words). However, we are not able to further generalize
9
3a
α1
α2
a+ K
b. N
n
s
aS
b
/
k
a
/
Fig. 4. Arbitrary delay scenario
ˆ is a language on alphaTheorem 5 to such Syn. For instance, in Theorem 5, Syn
bet (Σ − × Σ − ) ∪ {(a, a) : a ∈ Π}. That is, a symbol a must be synchronized to
the same symbol a. When a symbol can be synchronized with multiple symbols,
it is open whether the theorems obtained so far still hold.
Some other non-regular patterns Syn are also interesting. One such example
is that the Syn allows arbitrary delay between two synchronized symbols, which
is depicted in Figure 4. That is, Syn = {[w1 , w2 ] : w1 ↓Π = w2 ↓Π , w1 , w2 ∈ Σ ∗ }
where w1 ↓Π = w2 ↓Π says that the projections of w1 and w2 on synchronization
symbols Π are the same. This third pattern is called an arbitrary-delay Syn. Notice that this pattern excludes “unaligned” sequences like [eeaeeebee, ebeaeeeee],
where a, b ∈ Π and e 6∈ Π.
The covert communication processes explained earlier are an example of this
pattern in practice. Both processes can be thought of as asynchronous because
they are non-blocking and not time coupled. Although there is an arbitrary
amount of delay between CW sending a symbol and CR acknowledging the receipt
of the sent symbol, the processes could still be considered synchronous because
CW will not send another symbol until CR acknowledges the previous symbol.
The amount of delay between symbols could be nanoseconds or months.
We believe that the proof techniques used in Theorem 5 can be generalized
for the arbitrary-delay Syn and we conjecture
The mutual information rate in a synchronized regular counting language, specified in (7) with an arbitrary-delay Syn is computable.
There are also cases when the mutual information rate is not computable.
For instance, a synchronization mechanism results in a L12 whose emptiness
is undecidable (and hence its information rate is uncomputable). We will investigate some of these undecidable cases. Let M1 and M2 be two one-way
deterministic/non-deterministic automata. They work on two tapes separately
and independently. We use [w1 , w2 ] to denote the two inputs on the two tapes.
Notice that the lengths of the two words w1 and w2 are equal (hence one may
also think it is a single input tape with two tracks – M1 and M2 work on the
two tracks respectively and they don’t have to move in the same pace). However,
these two automata actually do not move completely independent. When one
reads a symbol a ∈ Π from its tape storing w1 , the other must also read the
same symbol a from its tape storing w2 , where a ∈ Π (Π ⊆ Σ is a given set of
10
synchronization symbols). In other words, w1 ↓Π = w2 ↓Π . Again, by assumption the lengths of w1 and w2 are the same. We say that [w1 , w2 ] is accepted
by [M1 , M2 ] if M1 and M2 accept w1 and w2 at the ends of the two inputs,
respectively. Here, L12 is the synchronized language accepted by [M1 , M2 ].
We now define more precisely how the one-way machines operate. We assume
that the input to a machine M has left and right end markers. At each step during
the computation on a given input w (which has left and right end markers, so the
input to M is actually #w$, where # and $ are the left and right end markers),
the machine’s input head can remain on the same cell or move one cell to the
right. (Note that the cell can contain #, $, or a symbol in Σ.) The input is
accepted if the machine eventually falls off the right end marker in an accepting
state. M is real-time if it does not have stationary moves, i.e., the input head
always moves one cell to the right.
Let M1 and M2 be two non-deterministic, one-way machines over input alphabet Σ. Let Σ = Π ∪ Σ − , where Π and Σ − are the (possibly empty) sets of
synchronizing and non-synchronizing symbols, respectively.
We say that M1 and M2 have a synchronized accepting computation on a
pair of inputs (w1 , w2 ) (the end markers are not shown) if the following holds:
1. |w1 | = |w2 |.
2. For i = 1, 2, Mi has an accepting computation on wi such that if at time t,
one machine is scanning a synchronizing symbol a, then the other machine
must also be scanning a at time t. Moreover, if on this synchronizing symbol
one machine makes a stationary move (i.e., remains on the same cell), then
the other machine must also make a stationary move.
If there is a synchronizing accepting computations on input pair (w1 , w2 ), we
use the notation [w1 , w2 ]. The set of all [w1 , w2 ]’s is denoted by L12 .
Theorem 7. It is undecidable, given two real-time 1-reversal deterministic pushdown automata (resp., real-time deterministic one-counter automata) M1 and
M2 , whether L12 = ∅.
But for reversal-bounded NCMs, we have:
Theorem 8. It is decidable, given two reversal-bounded NCMs M1 and M2 ,
whether L12 = ∅.
Corollary 1. We can effectively construct, given two reversal-bounded NCMs
(resp., DCMs) M1 and M2 , a 2-tape reversal-bounded NCM (resp., DCM) M
that accepts L12 .
For a set L ⊆ Σ ∗ × Σ ∗ , let R(L) = {x1 #x2 | (x1 , x2 ) ∈ L, and S(L) =
{x1 #xR
2 | (x1 , x2 ) ∈ L}.
Corollary 2. The information rates of R(L12 ) and S(L12 ) for reversal-bounded
DCMs M1 and M2 are computable.
11
A reversal-bounded NPCM is a non-deterministic pushdown automaton augmented with reversal-bounded counters. It is known that the emptiness, infiniteness, and disjointness problems for reversal-bounded NPCMs are decidable [13].
Corollary 3. It is decidable, given a reversal-bounded NPCM M1 and a reversalbounded NCM M2 , whether L12 = ∅.
The next result shows that under some restriction, emptiness of L12 is decidable when both machines are reversal-bounded NPCMs.
Let k ≥ 0. A machine M is k-sync if every input it accepts has at most
k-occurrences of synchronizing symbols on accepted inputs.
Theorem 9. Let k ≥ 0. It is decidable, given two k-sync reversal-bounded
NPCMs M1 and M2 on accepted inputs (there is no restriction in the occurrences of non-synchronizing symbols), whether L12 = ∅.
We can refine the proof Theorem 9. As we noted before, given two machines
M1 and M2 , we can think of the strings in L12 as a two-track tape, where the
first (resp., second) track is accepted by M1 (resp., M2 ).
Theorem 10. Let k ≥ 0. We can construct, given two k-sync reversal-bounded
NCMs (resp., reversal-bounded DCMs) M1 and M2 , a reversal-bounded NCM
(resp., reversal-bounded DCM) M accepting L12 .
Let M1 and M2 be k-sync reversal-bounded DCMs (i.e., the accepted strings
have at most k synchronizing symbols). We know from Theorem 14 that L12 is
accepted by a reversal-bounded DCM. We now show that the projection of L12
on the i-th coordinate (i = 1, 2) is accepted by a reversal-bounded DCM.
Lemma 1. Let M1 and M2 be k-sync reversal-bounded DCMs. Then Li = projection of L12 on the i-th coordinate is accepted by a reversal-bounded DCM for
i = 1, 2.
Lemma 2. Ls can be accepted by a reversal-bounded NCM Ms
Corollary 4. The mutual information rate of L12 for k-sync DCMs M1 and
M2 is computable, for any k.
4
Conclusions
In this paper, the mutual information rate is proposed as a quantitative method
for analyzing inter-process synchronized communication. For finite-state processes with implicit communication that can be described by a regular counting
language, the mutual information rate is effectively computable. When the synchronization always occurs between the same two symbols at the same time (or
with a fixed delay), the mutual information rate is also computable. Finally,
some cases are described where the mutual information rate is not computable.
12
References
1. E. Asarin, A. Degorre, Volume and entropy of regular timed languages: Discretization approach, in: M. Bravetti, G. Zavattaro (Eds.), CONCUR 2009 - Concurrency
Theory, Vol. 5710 of Lecture Notes in Computer Science, Springer Berlin Heidelberg, 2009, pp. 69–83.
2. R. Alur, D. L. Dill, A theory of timed automata, Theoretical Computer Science
126 (2) (1994) 183 – 235.
3. H. Chen, P. Malacaria, Quantitative analysis of leakage for multi-threaded programs, in: Proceedings of the 2007 Workshop on Programming Languages and
Analysis for Security, PLAS ’07, ACM, New York, NY, USA, 2007, pp. 31–40.
4. T. M. Cover, J. A. Thomas, Elements of information theory, 2nd Edition, WileyInterscience, 2006.
5. C. Cui, Z. Dang, T. Fischer, O. Ibarra, Information rate of some classes of nonregular languages: an automata-theoretic approach, in: Lecture Notes in Computer
Science, MFCS’14, Vol. 8634, Springer Berlin Heidelberg, 2014.
6. N. Chomsky, G. A. Miller, Finite state languages, Information and Control 1 (1958)
91–112.
7. C. Cui, Z. Dang, T. Fischer, Bit rate of programs, submitted.
8. C. Cui, Z. Dang, T. R. Fischer, O. H. Ibarra, Similarity in languages and programs,
Theor. Comput. Sci. 498 (2013) 58–75.
9. Z. Dang, O. Ibarra, T. Bultan, R. Kemmerer, J. Su, Binary reachability analysis
of discrete pushdown timed automata, in: E. Emerson, A. Sistla (Eds.), Computer
Aided Verification, Vol. 1855 of Lecture Notes in Computer Science, Springer Berlin
Heidelberg, 2000, pp. 69–84.
10. Z. Dang, O. Ibarra, Q. Li, Sampling a two-way finite automaton, in: Automata,
Universality, Computation, the series book: Emergence, Complexity and Computation, Springer, 2014.
11. Z. Dang, Pushdown timed automata: a binary reachability characterization and
safety verification, Theoretical Computer Science 302 (13) (2003) 93 – 121.
12. E. M. Gurari, O. H. Ibarra, The complexity of decision problems for finite-turn
multicounter machines, Journal of Computer and System Sciences 22 (2) (1981)
220–229.
13. O. H. Ibarra, Reversal-bounded multicounter machines and their decision problems, J. ACM 25 (1) (1978) 116–133.
14. O. H. Ibarra, Z. Dang, O. Egecioglu, G. Saxena, Characterizations of catalytic
membrane computing systems, in: B. Rovan, P. Vojt (Eds.), Mathematical Foundations of Computer Science 2003, Vol. 2747 of Lecture Notes in Computer Science,
Springer Berlin Heidelberg, 2003, pp. 480–489.
15. F. P. Kaminger, The noncomputability of the channel capacity of context-senstitive
languages, Inf. Comput. 17 (2) (1970) 175–182.
16. W. Kuich, On the entropy of context-free languages, Information and Control 16 (2)
(1970) 173–20.
17. E. Kushilevitz, Communication complexity, Advances in Computers 44 (1997) 331–
360.
18. Q. Li, Z. Dang, Sampling automata and programs, submitted (minor revision),
Theoretical Computer Science.
19. S. Muller, S. Chong, Towards a practical secure concurrent language, SIGPLAN
Not. 47 (10) (2012) 57–74.
13
20. G. Paun, Membrane Computing, an Introduction, Springer Berlin Heidelberg,
2000.
21. C. H. Papadimitriou, M. Sipser, Communication complexity, Journal of Computer
and System Sciences 28 (2) (1984) 260–269.
22. C. E. Shannon, W. Weaver, The Mathematical Theory of Communication, University of Illinois Press, 1949.
23. A. B. Shaffer, M. Auguston, C. E. Irvine, T. E. Levin, A security domain model to
assess software for exploitable covert channels, in: Proceedings of the Third ACM
SIGPLAN Workshop on Programming Languages and Analysis for Security, PLAS
’08, ACM, New York, NY, USA, 2008, pp. 45–56.
24. L. Staiger, The entropy of lukasiewicz-languages, in: Revised Papers from the 5th
International Conference on Developments in Language Theory, DLT ’01, SpringerVerlag, London, UK, 2002, pp. 155–165.
25. E. Wang, C. Cui, Z. Dang, T. R. Fischer, L. Yang, Zero-knowledge blackbox testing:
where are the faults? (International Journal of Foundations of Computer Science,
to appear).
26. G. Xie, Z. Dang, O. Ibarra, A solvable class of quadratic diophantine equations
with applications to verification of infinite-state systems, in: J. Baeten, J. Lenstra,
J. Parrow, G. Woeginger (Eds.), Automata, Languages and Programming, Vol.
2719 of Lecture Notes in Computer Science, Springer Berlin Heidelberg, 2003, pp.
668–680.
27. L. Yang, C. Cui, Z. Dang, T. R. Fischer, An information-theoretic complexity
metric for labeled graphs, submitted.
28. A. C. Yao, Some complexity questions related to distributive computing (preliminary report), in: Proceedings of the eleventh annual ACM symposium on Theory
of computing, ACM, 1979, pp. 209–213.
14
Appendix: Proofs
Proof of Theorem 5.
ˆ 1 and Q
ˆ2
Proof. For the L12 specified in (7), we assume that regular languages Q
are accepted by deterministic finite automata M1 and M2 , respectively. From
the definition in (4), it suffices to show the information rate λL2 (λL1 as well)
is computable. This is because, using Theorem 2, the information rate λL12
is computable due to the fact that L12 can be straightforwardly shown to be
accepted by a reversal-bounded DCM.
We first need some preparations.
We use Ls,a,s0 to denote the set of words accepted by DFA M1 when M1
starts from state s while a is the first symbol to be read, and, after further
reading zero or more symbols not in Π, enters state s0 and stops. Clearly, Ls,a,s0
is a regular language and hence semi-linear. By definition, the Parikh map of
Ls,a,s0 can be written as a finite number, say,
K(s, a, s0 ),
(8)
of linear sets. Such a linear set can be written in the form of
V = {v|v = v0 + b1 v1 + · · · + bt vt , bi ∈ N },
(9)
for some constant vectors v0 , v1 , · · · , vt , and some constant t. We use
T (s, a, s0 )
(10)
to denote the maximal t for all such linear sets in the Parikh map of Ls,a,s0 . The
linear set V in (9) naturally deduces a concatenated regular language, written
in the following regular expression,
α0 α1∗ · · · αt∗
(11)
where the Parikh maps of the words α0 , · · · , αt are exactly v0 , v1 , · · · , vt , respectively. In appearance, the regular expression has t + 1 segments where two
neighboring segments is separated by a separating point. There are at most t
separating points for a word τ in (11). Clearly, for the word τ , placement of the
separating points is not necessarily unique. One may also think of t separating
points as t distinct colors. In the sequence, we say the word τ is colored if the
first symbol of the segment in τ to the right of a separating point is further
decorated with the color of the separating point. There are at most t colored
symbols in τ . Again, for the same τ , the positions of colored symbols are not
necessarily unique. When the length m of τ is large (> |α0 |), there are at most,
using the definition in (10),
0
mT (s,a,s )
(12)
ways to color the symbols in τ .
Consider a synchronized word [w1 , w2 ] in L12 . Essentially, the word is a twotrack word. For a paired symbol (a, b) appearing in the word, we use (a, ·) for
15
the a in the first track and use (·, b) for the b in the second track. Since Syn
is 0-delay, the word [w1 , w2 ] is separated by synchronization pairs (a, a), with
a ∈ Π, and, paired symbols (on both tracks), called unsynchronized pairs, in
between two neighboring synchronization pairs, before the first synchronization
pair, or after the last synchronization pair, are pairs of symbols in Σ − (i.e., not
a synchronization symbol in Π). In this way, [w1 , w2 ] is cut into blocks as shown
in Figure 2:
– the first block consists of all the unsynchronized pairs before the first synchronization pair;
– the second block consists of the first synchronization pair, followed by all the
unsynchronized pairs before the second synchronization pair;
– and so on · · ·.
Let C be a large number (later on, we will take C to ∞). We say that a block
is small (resp. large) if its length is ≤ C (resp. > C). Consecutive small (resp.
large) blocks form a small (resp. large) cluster. Hence, the synchronized word
[w1 , w2 ] is a sequence of small clusters and large clusters, where a small cluster
and a large cluster appear alternatively.
The ultimate goal of the rest of the proof is to show that L2 is “accepted” by
some reversal-bounded DCM. A difficulty in the proof is as follows. For a w2 ∈
L2 , there are possibly many w1 satisfying [w1 , w2 ] ∈ L12 . Hence, there is inherent
non-determinism in guessing the right w2 to make the resulting [w1 , w2 ] satisfy
the Presburger constraint P . To do this, we artificially introduce determinism to
help choose the right w2 by modifying the w2 through the following decoration
process. The decoration on w2 in a [w1 , w2 ] ∈ L12 has two steps.
The first step is to decorate large clusters. Consider a (large) block in a large
cluster. The block is denoted by [au1 , au2 ], whose length is > C. Notice that
the block is a sub-word of [w1 , w2 ]. Suppose that, when the DFA M1 works on
w1 , its state is s right before it starts to read a in the au1 of the block. After
consuming the entire au1 , its state is s0 . In this way, au1 is in Ls,a,s0 defined
earlier. Hence, the Parikh map #(au1 ) of this au1 must be in one, say the i-th
(i is the index), of K(s, a, s0 ), defined in (8), number of linear sets. Suppose that
the linear set indexed by i is the one in (9). Therefore, #(au1 ) can be generated
from the Parikh map of a string τ in the concatenated regular language in (11).
Notice that the lengths of τ and of au1 are the same (since their Parikh maps are
also the same), and, after replacing au1 with τ in the block, τ still has the same
length as au2 and hence is aligned with au2 . As we have previously mentioned,
we suppose that τ is colored. That is, at most T (s, a, s0 ) symbols in τ are colored
so that the aforementioned separation points in (11) are explicitly indicated. In
fact, for a colored symbol e1 in τ , we decorate the same color on the symbol e2
in au2 that is right under the e1 in [τ, au2 ]. We use aˆ
u2 to denote the result. The
following properties are worth mentioning:
(P1). for each au2 , the following is true: there is an au1 with |au1 | = |au2 |
and au1 ∈ Ls,a,s0 iff there is a colored τ in (11) with index i and with
|τ | = |au2 |.
16
0
(P2). from (12), each au2 corresponds to at most mT (s,a,s ) distinct decorated aˆ
u2 , where m is the length of au2 .
We now further decorate the first symbol a in aˆ
u2 by a triple θ = (s, s0 , i),
recalling that i is the index mentioned above. The result is written as aθ u
ˆ2 .
Define constant K0 to be the maximum of
– the number of states in M1 ,
– maxs,a,s0 K(s, a, s0 ),
– maxs,a,s0 T (s, a, s0 ).
Consequently, there are at most K03 choices of the θ and, from property P2, there
are mK0 ways to color the au2 . Hence,
(P3). There are at most K03 mK0 distinct decorated aθ u
ˆ2 for the given
au2 (with length m) in a large block.
The second step is to decorate small clusters. A small cluster contains one or
more small blocks and the cluster starts with a synchronization pair, say (a, a).
Assume that the cluster is [au1 , au2 ]. The small cluster is decorated as follows.
There are two cases to consider.
Case 1. The total length of the small blocks in the cluster is ≤ C. That is, |au1 | =
|au2 | ≤ C. In this case, the cluster is called a C-group. Suppose that, when
the DFA M1 works on w1 , it is at state s right before M1 reads the first a
(in au1 ) and at state s0 when M1 reaches the end of au1 . We now decorate
the first a (in au2 ) with aπ where π = (s, s0 ). Obviously, since the number
of distinct π’s is bounded by K0 , we have the following property:
(P4). There are at most K0 distinct ways to decorate a small cluster
whose length m is ≤ C.
Case 2. The total length of the small blocks in the cluster is > C. Suppose that the
small blocks in the cluster are B1 , · · · , Bt for some t. Let t0 = 1.
– Identify the smallest t1 such that the total length of Bt0 · · · Bt1 is ≥ C;
the Bt0 · · · Bt1 is called the first C-group;
– Identify the smallest t2 such that the total length of Bt1 +1 · · · Bt2 is ≥ C;
the Bt1 +1 · · · Bt2 is called the second C-group;
– ···
– The last C-group is Btq−1 +1 · · · Bt , whose length is ≤ C, for some q.
The way to divide the cluster into C-groups is unique. Clearly, each C-group
has total length < 2C (since every block in the group is small) and the small
cluster has q groups with
(P5). q ≤ d m
C e.
Each C-group is decorated as follows. Consider a C-group [bv1 , bv2 ]. When
M1 works on w1 , M1 enters state s right before it starts to read the first b in
bv1 and enters state s0 when M1 reaches the end of the bv1 . We now decorate
the first b (in bv2 ) with bπ where π = (s, s0 ). Similar to property (P4), we
have, from (P5),
17
m
(P6). There are at most K0 d C e distinct ways to decorate a small
cluster whose length m is > C.
In summary, a small cluster is either a C-group by itself or a number of C-groups.
Combining (P4) and (P6), we have
m
(P7). There are at most K0 d C e distinct ways to decorate a small cluster
(regardless if its length m is > C or m is ≤ C).
We shall emphasize that the decoration is over the w2 in [w1 , w2 ]. The result
is written w
ˆ2 . We now construct a reversal-bounded DCM M to accept w
ˆ2 iff
there is a w1 with [w1 , w2 ] ∈ L12 . On reading a decorated w
ˆ2 , M simulates DFA
M2 with all the decorations ignored (i.e., M2 treats w
ˆ2 as w2 ). For each symbol
a ∈ Σ, we have a counter x(·,a) whose initial value is zero. Also, whenever M2
reads an a in w2 , x(·,a) is incremented by one. In parallel to M2 , M starts a
DFA M 0 that keeps track of all the decorated symbols in w
ˆ2 . When M 0 reads a
decorated symbol,
– If the decoration indicates that it is the start of a large cluster, M 0 uses its
finite memory to decode the decoration, which is a triple θ = (s, s0 , i) and
then starts to mimic reading a string τ in a concatenated regular expression
in (11). Recall that the regular expression is deduced from the i-th linear
set in the semi-linear set of Ls,a,s0 . We shall emphasize that the string is not
guessed. Instead, every separating point between two segments in the string
in (11) is already encoded in the colored symbols after the current decorated
symbol. Therefore, M 0 is deterministic. At the same time, for each symbol
a ∈ Σ, we have a counters x(a,·) , whose initial value is zero. Whenever M 0
reads an a in τ , x(a,·) is incremented by one.
– If the decoration says that it is the start of a C-group, M 0 uses its finite
memory to decode the decoration, which is a pair π = (s, s0 ). In this case,
suppose that the portion of w
ˆ2 that corresponds to the C-group is bπ v2 .
Notice that the length |bπ v2 | < 2C, and hence, since C is a constant, M 0
employs a counter ys,b,v2 ,s0 to count the total number of such decorated Cgroups. The total number of such counters, though many, is also a constant
on C. We now introduce some notations that will be used later. Recall that
in a C-group, there could be multiple small blocks. Therefore, inside v2 , there
still could be multiple synchronized symbols. Let bv1 be a string such that,
when M1 starts from state s, reading through bv1 and ends up in state s0 . In
ˆ
particular, [bv1 , bv2 ] are synchronized on Π; i.e., [bv1 , bv2 ] ∈ Syn.
Since the
length of bv2 is a constant, there are only finitely many such bv1 ’s; we use
Vs,b,v2 ,s0 to denote the set of Parikh maps of all such bv1 , which is a finite
set.
M 0 also checks that, using its finite memory, that the decorations are consistent in both states. For instance, in two neighboring C-groups decorated by
π1 = (s, s0 ) and π2 = (s00 , s000 ), M 0 must make sure that s0 = s00 . Also, the first
decorated symbol should have the s in the decoration as the initial state of M1
18
(without loss of generality, we assume that w2 starts with a synchronized symbol) and in the last decorated symbol should have the s0 in the decoration as an
accepting state of M1 .
M accepts when M2 enters an accepting state and M 0 verifies the state
consistency with decorations. However, so far, M only guarantees that there is a
ˆ1, Q
ˆ 2 ]∩ Syn.
ˆ
w1 satisfying [w1 , w2 ] ∈ [Q
We still need, using the counters in M , to
ensure that [w1 , w2 ] satisfies the Presburger constraint P in (7). The Parikh map
of w2 is stored in the vector X2 of counter x(·,a) , a ∈ Σ. Let X1 be the vector of
counters x(a,·) , a ∈ Σ. For every large cluster [bu1 , bu2 ], the Parikh map of the
first coordinate bu1 is accumulated in X1 . Therefore, the Parikh map of w1 is the
summation of X1 and Z, where Z is the accumulated Parikh maps of the first
coordinates in all the C-groups. For a C-group with its second-coordinate being
bv2 and decorated with (s, s0 ), the Parikh map of its first-coordinate is in the
finite set of vectors Vs,b,v2 ,s0 . Suppose that the vectors in Vs,b,v2 ,s0 are v1 , · · · , vq ,
for some q. The total number of all such C-groups is stored in counter ys,b,v2 ,s0 .
Hence, the accumulated Parikh map Zs,b,v2 ,s0 of the first-coordinates in all such
C-groups is exactly
∃z1 , · · · , zq ≥ 0, Zs,b,v2 ,s0 = z1 v1 + · · · + zq vq ∧ z1 + · · · + zq = ys,b,v2 ,s0 .
(13)
¯ s,b,v ,s0 , ys,b,v ,s0 ). There are
The Presburger formula in (13) is denoted by P(Z
2
2
0
only finitely many choices for s, b, v2 , s , since |bv2 | < 2C; we use Θ to denote all
the choices. Then, Z is definable by the summation of all the Zs,b,v2 ,s0 . That is,
There are Zs,b,v2 ,s0 , (s, b, v2 , s0 ) ∈ Θ, such that,
Z=
X
¯ s,b,v ,s0 , ys,b,v ,s0 ).
Zs,b,v2 ,s0 ∧ P(Z
2
2
(s,b,v2 ,s0 )∈Θ
ˆ
This is a Presburger formula, written P(Z,
Y ), where Y is the vector of all the
0
counters ys,b,v2 ,s0 , (s, b, v2 , s ) ∈ Θ. Finally, checking the Presburger constraint
P in (7) is equivalent to verifying
ˆ
∃Z, P (X1 + Z, X2 ) ∧ P(Z,
Y ),
that can be further written as a Presburger formula P(X1 , X2 , Y ) on counters in
X1 , X2 , Y of M . It is known that verifying counter values against a Presburger
formula can be done deterministically (with the help of finitely many axillary
counters), where all the counters are reversal-bounded. Hence, M is indeed a
reversal-bounded DCM.
M accepts a decorated version w
ˆ2 , instead of w2 with length n. There are
n
large blocks in [w1 , w2 ]; we use m1 , · · · , ml to list the lengths
totally at most C
of these
n
l≤
(14)
C
blocks. Clearly,
m1 + · · · + ml ≤ n.
(15)
19
Using (P3), there are a total number of Klarge of distinct ways to decorate these
large blocks, bounded above by
Y
Klarge ≤
0
K03 mK
i .
(16)
1≤i≤l
From (14) and (15), when n C, one can verify that Klarge in (16) can be
further bounded by
3n
Klarge ≤ K0C C
K0 n
C
.
(17)
In total, There are at most 2n
C C-groups in [w1 , w2 ]. Each C group has at most
K0 distinct ways to be decorated. Hence, the total number of distinct ways to
decorate all the small clusters is bounded above by
2n
Ksmall ≤ K0C .
(18)
Combining (17) and (18), every w2 corresponds to at most
3n
K0 (n, C) = K0C C
K0 n
C
2n
K0C
(19)
many decorated w
ˆ2 . Hence,
Sn (L2 ) ≤ Sn (L(M )) ≤ K0 (n, C) · Sn (L2 ).
(20)
Notice that the construction of the reversal-bounded DCM M actually implicitly
depends on C. For each given C, the information rate λL(M ) , denoted by λC , is
computable from Theorem 2. Denote KC = log K0n(n,C) = C5 log K0 + KC0 log C.
Notice that
lim KC = 0.
(21)
C→∞
From (20), we have
|λL2 − λC | ≤ KC .
(22)
The result follows, since λL2 is numerically computable up to any precision,
defined by KC .
Proof of Theorem 7
Proof. It is known that the disjointness problem (i.e., determining if L(M1 ) ∩
L(M2 ) = ∅) for these machines is undecidable. The result follows by making all
the symbols in Σ synchronizing symbols.
Proof of Theorem 8
20
Proof. Given M1 and M2 , we construct a 2-tape reversal-bounded NCM M
which simulates the machines in parallel using a separate set of reversal-bounded
counters for each machine. M only needs to make sure that the conditions above
are satisfied. To check that |w1 | = |w2 |, M uses two additional 1-reversal counters
to record the lengths of w1 and w2 and check that their lengths are the same at
the end of the computation. Clearly, L(M ) = ∅ if and only if L12 = ∅. The result
follows, since the emptiness problem for 2-tape NCMs is decidable [13] (this
follows from the observation that the projection of L(M ) on either coordinate is
accepted by a reversal-bounded NCM, and emptiness of reversal-bounded NCMs
is decidable).
Proof of Corollary 2
Proof. This follows from Corollary 1 and the known results that the information
rates of R(M ) and S(M ) for a a 2-tape reversal-bounded DCM is computable
[5].
Proof of Corollary 3
Proof. The construction in the proof of Theorem 8 applies, since the reversalbounded NCM M2 does not need the stack.
Proof of Theorem 9
Proof. We construct a 2-tape reversal-bounded NPCM M which, given input
(w1 , w2 ), simulates M1 and M2 sequentially as follows. M uses k additional 1reversal counters, c1 , . . . , ck . M also checks that |w1 | = |w2 | using two 1-reversal
counters.
M first simulates M1 on input w1 and records on counter ci the time ti when
M1 reads a new cell containing a synchronizing symbol ai for the first time. M
also records ai in the state. Note that c1 < c2 < · · · < ck .
When M1 accepts, M then simulates M2 and uses the counters ci to check
that M2 reads ai at time ti . When M2 accepts, M accepts. The results follows since we can check emptiness for 2-tape reversal-bounded NPCM (because
the projection of L(M ) on either coordinate is accepted by a reversal-bounded
NPCM.
Proof of Theorem 10
Proof. The idea is for M to simulate M1 and M2 in parallel on a given two-track
input string (a1 , b1 )(a2 , b2 ) · · · (an , bn ) (which has left and right end markers). So
the input to M (showing the end markers) will be
(#, #)(a1 , b1 )(a2 , b2 ) · · · (an , bn )($, $)
21
where convenient, (#, #) and ($, $) are interpreted as the the left markers #
and the right markers $.
In the simulation, if the input head of M is on a two-track symbol (α, β),
and both M1 and M2 remain (resp., move right) on α and β in a step, then M
remains (resp, move right) on (α, β). If one machine, say M1 , remains on α and
the other machine, say M2 , moves right on β in a step, M remains on (α, β)
while simulating the steps of M1 and delaying the simulation of M2 until M1
decides to move right of α. M operates this way until its input head reaches
the right end marker, ($, $). During the simulation, M uses two sets of counters
(c1 , c2 , . . . , ck ) and (d1 , d2 , . . . , dk ) to record the actual number of steps when M1
(resp., M2 ) first reaches the i-th synchronizing symbol on its track. (So when
the simulation of M2 is delayed for m steps while M1 is making m stationary
moves, m is added to ci but is not added to di . Note that c1 < c2 < · · · ck and
d1 < d2 < · · · dk . When M reaches ($, $), it verifies that ci = di for 1 ≤ i ≤ k.
Clearly, if M1 and M2 are reversal-bounded DCMs, then M is also a reversalbounded DCM.
Proof of Lemma 1
Proof. We only show it for L1 , the proof for L2 being similar. Let 0 ≤ r ≤ k
and s = (a1 , . . . , ar ) be an r-tuple of synchronizing symbols, which need not
be distinct (so, e.g., a1 = a3 = a6 , a2 = a5 , etc.). When r = 0, the tuple is
empty. Intuitively, the r-tuple s will be used to specify that an accepted input
has exactly the synchronizing symbols a1 , . . . , ar (in this order) appearing in the
input. (Recall that each input w of length n is provided with left end marker #
(at position 0) and right end marker $ (at position n + 1). Hence, the first and
last symbols of w are at positions 1 and n, respectively.)
Let Ls = {ai11 % · · · %aikr %bn | i1 , . . . , ir ≥ 1, there is a string of length n accepted by M2 in an accepting computation where the first time the input head
of M2 reaches the j-th synchronizing symbol is at time ij and that synchronizing
symbol at that time is ai . Thus i1 < i2 < · · · < ir . One can construct a reversalbounded NCM accepting Ls (see Lemma 2 below). Since Ls is a bounded language, we can convert this reversal-bounded NCM to a reversal-bounded DCM
Ms accepting Ls . So for each possible r-tuple s = (a1 , . . . , ar ), we construct a
reversal-bounded DCM accepting Ls . Let Qs = {(i1 , . . . , ir , n) | ai11 % · · · %aikr %bn ∈
Ls }. Note that Qs is semi-linear.
We then construct from M1 and the Ms ’s, a DCM M10 to accept L1 . The idea
is for M10 to simulate M1 while recording in new counters c1 , . . . cr , the first time
the input head of M1 reaches the i-th synchronizing symbol. M10 also records in
its finite control the r synchronizing symbols (in the order they appear) in the
finite control as s = (a1 , . . . , ar ) and the length of its input in another counter d.
After processing the input string, M10 then checks that the (r + 1)-tuple values
(i1 , . . . , ir , n) in counters c1 , . . . , cr , d is in Qs by simulating the appropriate
reversal-bounded DCM Ms .
Proof of Lemma 2
22
Proof. Given input ai11 % · · · %aikr %bn (not showing the end markers), M guesses
an input w symbol-by-symbol and simulates M2 while recording in counters
c1 , . . . , cr the first time ti it guesses the synchronizing symbol ai ( 1 ≤ i ≤ r). It
also uses another counter to record the length, |w|, of w. When M2 accepts, M
checks that the values in the counters c1 , . . . , cr , d are i1 , . . . , ir , n of the input.
Proof of Corollary 4
Proof. L12 can be accepted by a DCM from Theorem 14, and from Lemma1,
L1 and L2 (the projections of L12 on the first and second coordinates) can be
accepted by DCMs. The result follows since the information rate of the language
accepted by a DCM is computable.
23