View-invariant gait recognition system using a gait energy image decomposition method

: Gait recognition systems can capture biometrical information from a distance and without the user's active cooperation, making them suitable for surveillance environments. However, there are two challenges for gait recognition that need to be solved, namely when: (i) the walking direction is unknown and/or (ii) the subject's appearance changes significantly due to different clothes being worn or items being carried. This study discusses the problem of gait recognition in unconstrained environments and proposes a new system to tackle recognition when facing the two listed challenges. The system automatically identifies the walking direction using a perceptual hash (PHash) computed over the leg region of the gait energy image (GEI) and then compares it against the PHash values of different walking directions stored in the database. Robustness against appearance changes are obtained by decomposing the GEI into sections and selecting those sections unaltered by appearance changes for comparison against a database containing GEI sections for the identified walking direction. The proposed recognition method then recognises the user using a majority decision voting. The proposed view-invariant gait recognition system is computationally inexpensive and outperforms the state-of-the-art in terms of recognition performance.


Introduction
Traditional biometric traits, such as fingerprint, iris and so on are unique to a user, which explains why many applications rely on biometric systems for user recognition.However, these traits have a limited use in surveillance environments because they require active user cooperation and are difficult to acquire from a distance.To overcome this limitation, soft biometric traits, such as height, weight or gender have been explored, as they can be captured from a significant distance without the need for active user cooperation.The problem, however, is that these soft biometric traits are not distinctive enough to uniquely identify a user.Gait, on the other hand, which is the way a person walks, is a popular biometric trait that uniquely identifies a user from a distance [1].As such, gait is becoming an increasingly interesting trait to exploit in surveillance environments.
Many different methods explore gait for recognition, as discussed in [2].These include methods that rely on motion recording sensors worn on the body or force plates installed on the floor.Such methods are restricted to controlled environments only since the user/environment has to be set up with the recording device.However, there are also methods that simply use silhouettes obtained from video sequences.A popular example uses the gait energy image (GEI), computed from the silhouettes, for recognition [3].Other methods following similar strategies include [4][5][6], typically being developed to work on lateral walking direction sequences, and therefore having a limited application in unconstrained environments.
For gait recognition to be a viable alternative in unconstrained surveillance environments, the following two challenges have to be dealt with: • Unknown walking direction -When a user opts to walk in a random direction, the captured sequence cannot effectively be compared against database sequences captured from a different observation viewpoint.• Changes in subject appearance -When there are variations in the user's clothing or carried items, these appearance changes are not foreseen in the database.
In face of these challenges, the matching results may not be the desired ones.While in the literature one can find several proposals to tackle each of these challenges individually, only a limited number of proposals addresses both challenges simultaneously, as discussed below.

Related work
Gait recognition methods can be classified as either model-based or appearance-based recognition methods [1].
Model-based methods set parameters of three-dimensional (3D) models using the features obtained from the gait sequences.These models can then be used to synthesise images from the desired viewpoint, as presented in [7,8], or to extract new features to be used for user recognition, such as trajectories of key joints or lengths of segments between joints, as in [9].They also include methods that synthesise features for a desired viewpoint by using the perspective projection model and camera calibrations [10].Model-based methods usually perform extremely well in controlled environments, being robust to changes in the walking direction even within a gait cycle.However, since they typically require multiple cameras and additional setup information, including external and internal camera parameters or the position of the floor, they have a limited application in unconstrained surveillance environments.
For other model-based methods, gait is modelled by relying on features that are themselves invariant to changes in the walking direction.For instance, the work presented in [11,12] uses the hip, knee and ankle positions obtained from a gait sequence to model gait parameters.After a view rectification step, angular measurements derived over the gait cycle are used for user recognition.The method in [13] uses the head and feet positions obtained from the gait sequence to model their trajectories.The performance of these methods depends on the detection of invariant features and thus occlusions or the presence of artefacts in the images can hamper their performance.In addition, these methods are ineffective when there is a large change in the range of observed walking directions between testing and training sequences.
On the other hand, appearance-based methods use the spatiotemporal features obtained from observed gait sequences, without relying on any 3D model and its parameterisation.These methods perform gait recognition relying on sequences captured from a single camera, without the knowledge of any intrinsic and extrinsic camera parameters or depth cues.User recognition is typically performed by exploring inter-and intra-user correlations.View transformation methods, a sub-class of the appearance-based methods, generate gait feature vectors and a transformation matrix, typically by applying singular value decomposition on the GEIs for available walking directions.Thus, GEIs obtained from a particular walking direction can be transformed into another walking direction using only the transformation matrix.Examples include walking direction identification using linear discriminant analysis (LDA) [14], user recognition using multiple discriminant analysis (MDA) [15], and convolutional neural networks [16].The method presented in [17] is similar to [15] but uses a Radon transformbased energy image to improve recognition.The method presented in [18] obtains a transformation from any walking direction into a lateral walking direction by optimising low rank textures of a gait texture image.The transformation is applied to the gait silhouettes, which are then recognised using Procrustes shape analysis.Although effective, these methods are applicable when the walking direction changes within a limited range around the lateral walking direction.Since the transformation degrades when the walking direction changes a lot with respect to the trained walking direction, poor recognition results are obtained in those conditions.In addition, as the methods rely on inter-and intra-user correlation, further appearance changes to the GEIs, e.g.due to bags or coats, will adversely affect their performance.Some appearance-based methods tackle the identified limitations by splitting gait recognition into two steps.They first identify the walking direction of the user, and then perform user recognition for the identified walking direction.This two-step approach restricts compared to the identified walking direction, of course implying that users have to be registered in all the considered directions.The methods presented in [19,20] use the GEI's leg region entropy to identify the walking direction, followed by random subspace learning (RSL) for user recognition.The method in [19] further improves robustness to appearance changes by Gaussian filtering the GEI at different scales, generating a multiscale gait image that gradually highlights the subject's shape, which is unaltered by appearance changes.The method presented in [21] trains the Gaussian process (GP) classifier using GEI's leg region for walking direction identification and performs user recognition using canonical correlation analysis (CCA).In [22] walking direction identification is performed by analysing the feet positions in a GTI and user recognition is performed by applying LDA to dissimilarity vectors which represent a user.Although these methods are effective, some operate over a limited range of walking directions, while some others provide a limited robustness against appearance changes and walking direction identification.In addition, most of the mentioned methods are computationally expensive.
As expected, methods that operate only with lateral walking direction report increased robustness to appearance changes.In this observation viewpoint, methods explore different gait representations to obtain better recognition results.The work in [23] uses Poisson random walk as a gait feature, while [24,25] use gait entropy image (GEnI).The work presented in [26] selects the common parts of the test GEnI and the training GEnIs, including those altered by appearance changes, by using a binary mask obtained from the two GEnIs being compared.The mask thus eliminates the alterations caused by the appearance change before applying MDA for user recognition.The method presented in [27] sets higher weights to areas unaltered by appearance changes, and sets lower weights to altered areas during the recognition step.A golden ratio-based approach, in [28], uses four different clothing models to identify unaltered area of the test GEI that is used for recognition.Although these methods handle appearance changes, most are limited by the use of information about the type of appearance change considered while training.

Motivation and contribution
From the above discussion, it can be concluded that appearancebased methods are better suited for surveillance environments, since they can deal with changes in the walking direction with fewer constraints.Among appearance-based methods, those that are more effective against appearance changes rely on two steps, first identifying the walking direction, and then performing user recognition.Here, there is room for improving both the walking direction identification and the overall recognition performance.
This paper presents a system comprising two novel methods: one for walking direction identification and the other for user recognition when subjected to appearance changes, achieving better walking direction identification and user recognition results when compared to the state-of-the-art.The proposed system follows the two-step approach described above.Walking direction identification is performed by computing PHash over the leg region of the GEI.Test and training PHash values are compared using hamming distances, making the method computationally inexpensive.User recognition is then performed by the system using the novel method of GEI decomposition into sections.Only the GEI sections that are unaltered by appearance changes are selected for the matching step.Unaltered sections are identified by analysing the difference between the average of all training GEIs available and the test GEI.Sections are separately matched against the user database corresponding to the identified walking direction.User recognition relies on a majority voting decision, among the selected GEI sections.Since the appearance changes caused by a coat or a bag do not alter the entire GEI, its decomposition allows a good match between the unaltered GEI sections, making the proposed method perform better against appearance changes when compared to available state-of-the-art methods.
The major contributions of this paper can be summarised as follows: • Walking direction identification -The paper improves on the work presented in [29], obtaining better walking direction identification results, achieved by better adapting the PHash method for walking direction identification.• User recognition -A novel recognition method is proposed that decomposes the test GEI into sections.Analysing the difference between the test and the training GEIs average allows selecting those sections unaltered by appearance changes.The method obtains better user recognition results when compared to the state-of-the-art.It is also robust to walking direction misclassifications among neighbouring walking directions.
The remaining of this paper is organised as follows.Section 2 presents the proposed system, with the corresponding experimental results being reported in Section 3. Section 4 provides conclusions and directions for future work.

Proposed system
The proposed system architecture is presented in Fig. 1, consisting of two phases: training and testing.In the training phase, the user GEI and walking direction features considered of interest are recorded in the database.The walking direction database holds the PHash values computed over training GEI's leg region associated with the considered walking directions.For the recognition step, the GEIs corresponding to each walking direction are grouped together.They are further decomposed into horizontal sections of equal size, followed by the application of principal component analysis (PCA) and LDA for dimensionality reduction and data decorrelation.The obtained features are recorded such that each walking direction has a separate user database.Thus, the performance of the proposed system depends on the range of walking directions available for training.However, since the proposed user recognition method is robust to slight changes in the detected walking direction, the considered directions need not be extremely granular.
Once training is completed, the system performance can be evaluated with a disjoint set of test sequences.Walking direction identification is performed using k-nearest neighbour (k-NN) with Hamming distance as the distance measure between the test and training PHash values.For user recognition, the test GEI is decomposed into sections, following the procedure described for the training step.Sections unaltered by appearance changes are selected by analysing the difference between the test and the training GEIs' average and a matching score is obtained for each of them.Finally, the system adopts a majority voting decision for user recognition.

Walking direction identification
The proposed method, instead of training with each user's leg region, as done in [19,20,26], trains with the general shape of the leg region, computed from all users for each considered walking direction.The general shape of the leg region is obtained by computing a PHash over approximately the bottom third part of the GEI.Unlike with cryptographic hash methods, perceptual hash outputs can be compared to obtain a distance measure.The first step in [29] suggests resizing the leg region to a square block of 32 × 32 pixels, for reducing the computational complexity.However, the size of the leg region obtained is not significantly larger in the already size-normalised GEIs.Resizing also affects the overall shape of the leg region causing misclassification among neighbouring walking directions.Thus, in this paper, it is proposed to skip the resizing step.Next, the discrete cosine transform (DCT) is computed over the leg region of the GEI.The lower frequency DCT coefficients represent the most relevant shape information, allowing to discard higher frequency components to obtain a more compact descriptor -details on coefficient selection are given in Section 3.1.The unidimensional PHash directional descriptor results from concatenating the selected coefficients using a raster scan order.Then PHash can be converted to a binary vector, according to the following equation: where PHash I c and DCT I c are, respectively, the PHash bit values and the DCT coefficient at position I c within the selected lower frequency DCT coefficients.
the mean of these DCT coefficients.The steps for PHash computation are illustrated in Fig. 2. The obtained PHash values tends to be similar as long as the overall shape of the GEI's leg region remains similar, making it ideal for walking direction identification.As described above, the PHash obtained from the training GEIs are stored in the corresponding walking direction database and are compared to each test sequence PHash values, using k-NN with Hamming distance as the distance measure, to identify the walking direction.

User recognition
Once the walking direction is identified, user recognition can be performed by matching the test GEI against the database containing the training GEIs corresponding to the identified walking direction.Matching GEIs obtained from different walking The proposed method decomposes the GEI into N horizontal sections of equal size.The rationale behind splitting the GEI is that a bag or a coat does not alter the appearance of the entire GEI but only a small section of it.This small section of the GEI is usually responsible for the poor recognition results, as sections altered by appearance changes are seldomly matched to the correct user.By decomposing the GEI into sections, the effect of change in appearance is limited to only a few of the sections, allowing the rest of the sections to be successfully matched to the correct user registered in the database.
Next, the rows of each GEI section are concatenated (from top to bottom) to form a unidimensional vector.The vectors for each section, from all users registered in the database, are stacked to create a matrix.Its dimensionality is reduced by applying PCA, selecting the principal components with the highest variance and obtaining its projection onto the selected components.
For each section, LDA is employed for data decorrelation, by identifying a projection matrix onto a subspace that maximises the ratio of intra-to inter-class scatter, using the Fisher's criterion.Given n classes, the intra-class scatter matrix Σ w is given by (2)  and the inter-class scatter matrix Σ b is given by (3).
where x ¯i is the mean of the class c i , m i is the number of training samples for each class c i and x ¯ is the total mean (considering all classes).The transition matrix ϕ that maximises the ratio of the between-class scatter matrix to the within class scatter matrix is given below: To perform recognition over a test GEI the system decomposes it into N horizontal sections of equal size and selects the sections that are unaltered under appearance changes.To do so, one possibility is to create a binary mask that selects similar parts between a test and a training GEnI [26].However, individual user's appearance can affect its performance.To overcome that problem, the proposed method computes an average GEI over all training sequences corresponding to each walking direction.The average image represents the general shape of the unaltered database users with respect to a walking direction.The unaltered sections in a test GEI can then be selected by applying a threshold T to the difference between the average image and the test GEI, with the differences in the altered sections being significantly larger than what is observed for unaltered sections.The threshold T is empirically set to 150.The selected sections can then be projected onto the selected principal components.
Given a test GEI section z, its classification into one of the existing classes (i.e.users registered in the database) is performed in the transformed space, based on Euclidean distance d(, ), according to the following eqation: where x ¯k is the centroid of the kth class.Next, user recognition is performed by a majority voting decision among the selected sections.Each section votes for the user class that it is classified into, and the test GEI is classified into the user class receiving most votes.For the example illustrated in Fig. 3, the leg sections of the GEI, which are unaltered by the coat, are correctly classified into their user class (in this case 1), while the sections that are altered by the coat are assigned to apparently random classes.In this example, the correct user is identified with the majority of the votes.

Experimental results
The proposed system is tested using the dataset B of CASIA Gait Database, collected by the Institute of Automation of the Chinese Academy of Sciences [30] and the Large Population Dataset of the OU-ISIR Gait Database [31].The dataset B of CASIA Gait Database contains gait silhouette sequences of 124 users, captured for 11 different walking directions, with angles of acquisition ranging from 0° to 180°, with a step of 18° between adjacent views.For each angle, there are ten sequences per user, out of which six correspond to normal walking, in two of them the user wears a coat, and in other two the user carries a bag.
The Large Population Dataset of the OU-ISIR Gait Database consists of 4016 users.It contains four walking directions for each user (55°, 65°, 75° and 85°).It should be noted that all four walking directions are derived from the same video sequence, such that for each of them there is one gait cycle available, and the walking direction changes along the gait cycle.Moreover, the provided GEIs have undergone a perspective correction.Each walking direction has one sequence for training and one for testing.

Walking direction identification results -CASIA Gait Database
Before performing walking direction identification, some initial tests are conducted to adjust PHash parameters: (i) select the percentage of the GEI to be used as the leg region; (ii) the number of DCT coefficients to use; and (iii) the value of parameter k to be set for k-NN.The tests are conducted by varying these parameters over a range of values.The best walking direction identification results are obtained using the bottom 33% of the GEI to represent the leg region, the 22 × 22 lower frequency DCT coefficients to generate the PHash and the value of k is set to 6.These are the default values for the conducted experiments.
To evaluate the performance of the proposed PHash method it is compared, in Table 1, to the state-of-the-art methods reported in [19,20,22,26], as they are applicable in the cases where the users are wearing a coat or carrying a bag.The entropy method [19] is evaluated similarly to the proposed PHash method, using the first four normal sequences for training and the remaining two normal, two coat and two bag sequences for testing.The improved entropy method [20] considers three normal sequences for training, while the remaining sequences are used for testing.The method in [22] uses all the sequences for testing.The GP method [26] is computed for a subset of seven walking directions (36°-144°), with the training consisting of four normal sequences for 60% of the total users available, the remaining 40% being considered for testing, and thus not reported in Table 1.Adopting the setup of the GP method setup, the proposed PHash method obtains a mean walking direction identification score of 95% for normal sequences, 94% for coat sequences and 93% for bag sequences.
Table 1 displays the obtained walking direction identification results where each row represents the walking direction.Each method has three columns associated to it where the first corresponds to normal sequences while second and third corresponds to sequences altered by coats and bags, respectively.From the mean results, it can be concluded that the proposed PHash method performs better than the state-of-the-art methods, with an average correct walking direction identification rate of 97%.The significant improvement in the performance can be attributed to the use of the PHash.It extracts the shape information of the leg region making it robust to alterations caused by bags and coats, a factor that hampers the performance of most state-of-theart methods.Although the Contour Method [22] performs equally well it involves the construction of a new gait feature called GTI, which needs additional information from the environment thus restricting its use when, as observed for the Osaka database, only cropped silhouettes and not the entire image are available.A second advantage is that most misclassifications usually occur into neighbouring walking directions as shown in Table 2, and misclassifications between neighbouring walking directions do not significantly affect the recognition performance of the system, as discussed in Section 3.3.
A third advantage of the proposed PHash method lies in its low computational complexity.The method is compared to a reimplementation of the entropy method proposed in [20], using a computer with an Intel(R) Core(TM) I7@3.60 GHz CPU, with 32 GB of RAM, in both cases running the same MATLAB R2014b code, just replacing the feature selection and distance computation (either PHash or the entropy method).The proposed PHash method performs the computations, for a user in Table 1, with an average time of 0.02 s, compared to 0.3 s required by the entropy method.The PHash is extremely simple and easy to compute, making it significantly faster than the entropy method.

User recognition results -CASIA Gait Database
To show the effectiveness of the proposed GEI decomposition method, it is compared to the state-of-the-art methods that are robust to appearance changes for lateral walking (90°) sequences.The method uses the first four normal sequences for training and the remaining two normal, two bag and two coat sequences for testing.Results are reported in Table 3.A preliminary test has been conducted to identify the number of ideal sections N the GEI can be decomposed into.The number of sections varied from 5 to 20.It can be observed in Fig. 4 that optimal number of sections is 13 for coat sequences and 11 for bag sequences, while the performance for normal sequences is indifferent to the number of sections.The default value for the number of sections is set to 11, since it provides the best mean performance across all lateral walking sequences.

Fig. 3 Example of proposed user recognition method (a) Average image created using database GEIs for a given walking direction, (b) GEI of a 'normal' walking sequence, (c) GEI of a walking sequence altered by the user wearing a coat, (d) Binary mask representing the thresholded difference between a and c, (e) GEI sections of the walking sequence in c, (f) GEI sections selected for recognition
Table 1 Correct walking direction identification rate (%) View (deg.)GP method [26] Entropy method [19] Improved entropy method [20] Contour method [22] Proposed PHash method The results in Table 3 show that the mean performance of the proposed GEI decomposition method is better than the state-of-theart methods for lateral walking gait recognition.Most of the methods are very effective for normal waking sequences with a correct classification rate of almost 100%.The challenge, however, arises in face of appearance changes.It is observed that the performance of the proposed method is far better than the state-ofthe-art methods when the subjects wear coats, while its performance when carrying bags is among the best listed.The decomposition of the GEI into N sections allows isolation of the sections altered by appearance changes, thus improving performance.Even without the selection of unaltered sections, the proposed method obtains competitive performance, with a correct recognition rate of 94 and 86% for coats and bags, respectively.

Proposed system results -CASIA Gait Database
The next set of results reflects the integration of the proposed PHash and GEI decomposition methods.To show the effectiveness of the proposed system, the entire CASIA B Database is used.The first four normal sequences of each user are used for training, while the remaining two normal, two bag and two coat sequences are used for testing.The test GEIs are initially sorted with respect to their walking direction using the proposed PHash method, which is then followed by the proposed GEI decomposition method for user recognition.
Correct user recognition rate results are shown in Table 4.As expected, very good results are obtained for normal walking sequences.It is worth noting that a correct walking direction identification rate of 97%, as reported in Table 1, in the case of normal sequences leads to a correct user recognition rate of 99%see Table 4. Thus, it can be concluded that even in some cases where the walking direction is incorrectly identified, the proposed GEI decomposition method performs correct user recognition, being robust to small walking direction misclassifications.The GEI decomposition method performs best for near lateral walking directions with a correct recognition rate of 95% for 90° in the case of coat sequences, as the appearance change is easy to detect for near lateral walking directions, as shown in Figs.5c and d.The appearance change caused by bags is easy to detect and is more localised for all walking directions as shown in Figs.5e and f.To show the effectiveness of the proposed GEI decomposition method in user recognition and robustness to walking direction misclassification, a second set of result is obtained where the walking direction is ideally sorted before user recognition.It can be seen from Table 4 that in this ideal case the proposed method's performance is almost equivalent to when walking direction is identified using PHash method.Thus, it can be concluded that the proposed method is robust to walking direction misclassification.
The proposed system is compared to the state-of-the-art given in Table 5.It displays its robustness to appearance changes, especially in the case of coat sequences, where it outperforms its competitors.It should be noted that the proposed system is    indifferent to the type of appearance change, while the multiscale method presented in [19] uses the information about the type of appearance change to adapt to the various situations.That method chooses the scales for its multiscale Gaussian method by considering the three cases of normal, bag and coat separately, thus making its application more constrained.

Walking direction identification results -OU-ISIR Gait Database
To test the performance of the proposed walking direction identification method over the OU-ISIR Gait Database the protocol presented in [16] is followed.According to the protocol, a subset of 1912 users is selected.The subset is then divided into two disjoint groups of 956 users each.The first group is used to train and the second group is used to test the proposed walking direction identification method.The final result is obtained by repeating the test for ten different subsets.
To perform these tests the 22 × 22 lower frequency DCT coefficients are used to generate the PHash and the value of k is set to six for k-NN, as discussed in Section 3.1.Since the silhouettes of the OU-ISIR Gait Database are normalised to compensate for the walking direction change, the shape information in the leg region of the GEI is distorted and thus the bottom third of the GEI (leg region) produces poor results.Therefore, for this database the shape of the entire GEI is considered for computing the PHash, allowing to achieve a correct walking direction identification rate of 80%.The confusion matrix for these results is included in Table 6, which shows that most misclassification errors occur with the neighbouring walking directions.The appearance of the GEIs for 75° and 85° are visually very similar, which leads to poor results in distinguishing them, while the GEIs for 55° appear as much more distinct.The proposed method exploits this visual difference using PHash.

User recognition results -OU-ISIR Gait Database
Since the proposed recognition method uses LDA and the OU-ISIR Gait Database provides only one training sequence for each user, user recognition cannot be performed following the protocol presented in [16].To overcome this limitation, the GEIs of the four available walking directions in the training set are used as if they all corresponded to the same walking direction, to allow performing the tests with the methodology presented in Section 3.2.Moreover, since the size of the GEIs is small, the number of sections, N, is limited to 6, to prevent sections from being extremely small which would hamper performance.In these conditions, the proposed system achieved a recognition rate around 97%, which is a significant improvement from the 94% reported in [31].
A second test is conducted where the training only considers directions different from the one that is being tested.This is repeated for all four walking directions.The result is an average correct user recognition rate of 94%, which shows the robustness of the proposed method to walking direction misclassification.Since the OU-ISIR Gait Database is not often used in the literature for testing walking direction identification or appearance changes, the results could not be compared to the state-of-the-art.

Conclusion
This paper proposes a new gait recognition system that tackles change in walking direction and change in appearance by addressing them in two steps, handling a single problem at a time.
Step 1 performs walking direction identification by computing a PHash over the leg region of the GEI.The obtained PHash is matched against the database using k-NN with Hamming distance to identify the walking direction.As for Step 2, it performs user recognition by decomposing the GEI into sections and selecting the sections unaltered by appearance changes.Each section is then individually matched against the corresponding sections registered in the user database, using LDA.The final classification is performed using a majority voting decision.The decomposition of the GEI into sections limits the influence of appearance changes to a few sections, while analysing the difference between a test GEI and the average of all available training GEIs helps identify and discard the altered sections, making the system robust against such changes.This allows the proposed system to outperform the stateof-the-art.
The proposed system uses the GEI representation of gait together with simple classification methods, such as LDA.Therefore, future work will consider exploring more complex classification tools to improve user recognition.Furthermore, the work will also include improving the walking direction identification method to tackle changes within a gait cycle by further exploring features that are unaltered by appearance change and thus better reflect the user's walking direction.

Fig. 4
Fig.4 Performance of the proposed decomposition method with respect to the number of sections N

Fig. 5
Fig. 5 Lateral and frontal examples of GEIs for (a, b) Normal, (c, d) Coat and (e, f) Bag sequences

Table 2
Confusion matrix of the proposed method for the results in Table 1 (number of sequences)