|
|
||||||||
1From the Institute of Biomedical Research, MRC Centre for Immune Regulation, Division of Immunity and Infection, Medical School, The University of Birmingham, Birmingham, United Kingdom; 2Academic Unit of Ophthalmology, Division of Immunity and Infection, University of Birmingham, Birmingham and Midland Eye Centre, City Hospital NHS Trust, Birmingham, United Kingdom; and 3School of Biosciences, The University of Birmingham, Birmingham, United Kingdom.
| Abstract |
|---|
|
|
|---|
METHODS. AqH from noninflammatory control subjects and patients with idiopathic, Fuchs heterochromic cyclitis (FHC), and herpes-viral or Behçets uveitis were analyzed for IL-1ß, -2, -4, -5, -7, -8, -10, -12, -13, -15, TNF
, IFN
, CCL2 (MCP-1), CCL5 (RANTES), CCL11 (Eotaxin), TGFß2, and CXCL12 (SDF-1), using multiplex bead immunoassays. The cellular infiltrate was also determined for each sample.
RESULTS. Idiopathic uveitis AqH, compared with noninflammatory controls, was characterized by high levels of IL-6, IL-8, CCL2 and IFN
, the levels of which correlated with each other. For IL-6 and IL-8 these levels were proportional to the number of neutrophils present. By contrast, the levels of both TGFß2 and CXCL12 decreased in idiopathic uveitis AqH with increasing inflammation. Cluster analysis showed a degree of segregation between noninflammatory and idiopathic uveitis AqH. Further examination using random forest analysis yielded a complete distinction between these two groups. The minimum cytokines required for this classification were IL-6, IL-8, CCL2, IL-13, TNF
, and IL-2.
CONCLUSIONS. Application of multiplex bead immunoassays has allowed us to identify distinct patterns of cytokines that relate to both clinical disease and the cellular infiltrates present. Bioinformatics analysis allowed identification of cytokines that differentiate idiopathic uveitis from noninflammatory control AqH and are likely to be important for the pathogenesis of uveitis.
), CCL4 (MIP1ß), CCL5 (RANTES), and IFN
have all been reported.6 8 11 12 14 15 16 17 18 19 These studies correlate well with experimental uveitis where, during the course of experimental autoimmune uveoretinitis and endotoxin-induced uveitis, levels of IL-6, TNF
, IL-1ß, CCL2, CCL3, CCL5, and IFN
increase with disease.20 21 22 23 24 Despite these studies, examination of the complex patterns of these molecules in human uveitis has been limited by the quantity of AqH available from each patient. For most studies, comparisons were made using either a few samples for each cytokine or by using a different cohort of patients for each molecule of interest. This has resulted in inconsistent reports in the literature for a number of cytokines, including IL-12 and IL-10,8 10 11 12 25 and these studies have been unable to analyze the complex relationships between the cytokines and clinical disease. Recently, multiplex-bead-based immunoassays have been established that allow the identification of many molecules in a single small sample volume.26 27 They have already been successfully used to measure cytokines in serum, cerebrospinal fluid, tears, and culture supernatants,28 29 30 31 32 and this technique is ideally suited to the analysis of AqH. By applying conventional statistical tests to the multiplex data, it may be possible to identify cytokines and chemokines that are differentially expressed between two groups of patients (e.g., uveitis vs. noninflammatory controls). However, this approach examines only one molecule at a time and does not take into consideration that a combination of cytokines may be required to adequately describe the differences between two groups of patients. Methods to identify multimarker predictors of disease status have been developed and applied in a large number of clinical studies. In this study we have used a recently developed method, random forest analysis.33 This method is based on the principal of decision trees and incorporates efficient methods to establish the importance of each variable in the classification and to perform an unbiased estimate of the classification error. Converting the structure of the trees into a two-dimensional model allows groups of patients to be visually represented in a graphical manner.
The aim of our study was to undertake multiplex immunoassay analysis of uveitis AqH for cytokines and chemokines and to apply both traditional and cluster analysis to the data. The ability to measure numerous molecules in a single sample and to perform complex analysis may allow a greater insight into the intraocular environment during active uveitis and underlying pathogenetic mechanisms.
| Methods |
|---|
|
|
|---|
AqH was centrifuged at 300g for 5 minutes, the cell-free supernatant (subsequently referred to as AqH) was removed and frozen in aliquots at 70°C. Cell pellets were resuspended, counted, and cytocentrifuged (Cytospin; Shandon Inc., Pittsburgh, PA). Slides were stained with Diff-Quik (Dade Behring, Marburg, Germany) and viewed by light microscopy for determination of the number and the percentage of lymphocytes, macrophages, and neutrophils.
Purification of CD4+CD45RA+ and CD45RO+ Peripheral Blood T Cells
Naïve (CD45RA+) and primed (CD45RO+) CD4+ T cells were isolated from the peripheral blood of healthy volunteers. Peripheral blood mononuclear cells isolated by density gradient centrifugation were depleted of adherent cells by two rounds of incubation on plastic at 37°C for 30 minutes each. Nonadherent cells were incubated for 30 minutes at 4°C with a combination of antibodies specific for CD8, CD11b, CD14, CD16, CD19, TCR
, glycophorin A, and either CD45RA or CD45RO. After washing, the cells were incubated for 30 minutes at 4°C, with continuous rotation, with sheep anti-mouse IgG Dynabeads (Dynal Biotech, Wirral, UK). The bead depletion was repeated to produce CD4+CD45RA+ and CD4+CD45RO+ populations of >95% purity (typically >98%).
Cytokine Measurement in AqH
AqH samples (50-µL volumes) were analyzed using multiplex bead analysis that uses microspheres as the solid support for immunoassays and allows the analysis of all molecules from each sample.27 IL-1ß, -2, -4, -5, -8, -10, -12, -13, TNF
, IFN
, CCL2, CCL5, and CCL11 were measured according to the manufacturers instructions (Upstate Biotechnology UK, Buckingham, UK). IL-7, IL-15, TGFß2, and CXCL12 were measured using a multiplex bead immunoassay developed in the laboratory. Paired anticytokine antibodies, already tested for use in ELISA (R&D Systems, Abingdon, UK), were selected, because they have already been extensively tested for cross-reactivity to other cytokines. Carboxylated xMAP fluorescent microbeads (Luminex Corporation, Austin, TX) were centrifuged, the pellet sonicated, and 100-µL beads (1.25 x 106) were activated with 5 mg/mL N-hydroxysulfosuccinimide sodium salt and 5 mg/mL 1-ethyl-3-(3-dimethylaminopropyl) carbodiimide hydrochloride in 0.1M NaH2PO4, pH 6.2, for 20 minutes at room temperature. Beads were centrifuged and washed twice with PBS. Monoclonal capture antibody (50 µg) was added in a volume of 250 µL for 2 hours at room temperature. Antibody-coupled beads were washed 3 times with PBS 0.05% Tween-20 and were resuspended in PBS, 1% BSA, 0.05% NaN3. Successful coupling of the capture antibody to the beads was assessed by staining with FITC goat anti-mouse IgG (Southern Biotech, Birmingham, AL), followed by flow cytometric analysis. Each antibody was coupled to a different fluorescent bead. Cross-reactivities in the multiplex assay were rigorously tested and were always <0.01%.
For cytokine measurements, AqH samples, diluted with an equal volume of PBS, 1% BSA, 0.05% Tween 20, were incubated with monoclonal antibody-coated capture beads (2 x 103 for each cytokine) for 2 hours at 20°C. Washed beads were further incubated with biotin-labeled polyclonal anti-human cytokine antibody (2 µg/mL) for 2 hours, followed by streptavidin-phycoerythrin (Upstate Biotechnology UK) at 40 µg/mL for 30 minutes. Samples were analyzed using a microbead analyzer (Luminex 100; Luminex Corporation). Standard curves of known concentrations of recombinant human cytokines (R&D Systems) were used to convert fluorescence units to cytokine concentration (pg/mL).
Statistical Analysis
Dunns multiple comparison tests were used to compare the levels of each cytokine measured between the noninflammatory control, idiopathic, FHC, and herpes-viral and Behçets uveitis groups. Spearman (two-tailed, nonparametric) correlations were used to assess the significance of correlations between each cytokine and between the cytokines and cellular composition of the AqH. This was only carried out for the idiopathic group. The level of confidence at which the results were judged significant was P < 0.05.
Cluster and Random Forest Analysis
Clustering was performed using average linkage on a similarity matrix derived using Pearson moment correlations. The analysis was performed using the Web-based toolset GEPAS. To develop classification models, we have used a recently developed tree classification algorithm, random forest.33 This method grows a collection of decision trees constructed using a different bootstrap sample of the data. Each node is split, using the best among a subset of predictors randomly chosen at that node.
Importance of Individual Cytokines in the Classification
The random forest algorithm estimates the importance of a variable by looking at how much prediction error increases when data for that variable is permuted while all others are left unchanged. The necessary calculations are carried out tree by tree as the random forest is constructed. In this study we have determined the smallest number of cytokines that could be used in each classification.
Visualization of the Models
The models developed using random forest can be visualized graphically using a representation of the collection of trees. The first step is to produce a proximity matrix that represents the relative similarity between samples as represented in the trees. This is achieved as follows: the (i, j) element of the proximity matrix produced by random forest is the fraction of trees in which elements i and j fall in the same terminal node. The idea is that data points (patients, in our case) that are similar to one another will frequently end up in the same terminal node of a tree; exactly what is measured by the proximity matrix. Thus, the proximity matrix can be taken as a similarity measure, and multidimensional scaling using this similarity can be used to divide the original data points into groups for visual exploration.
| Results |
|---|
|
|
|---|
, and IFN
were all detectable at high levels (Fig. 1B) . These cytokines were only detected after anti-CD3/28 stimulation. IL-8 and CCL5 could be detected from both naïve and primed CD4+ T cells. The proliferation of the two subsets of cells was equivalent. These results demonstrate that multiplex bead immunoassays can be used to identify distinct cytokine profiles in single small sample volumes.
|
, IFN
, GM-CSF, CCL2, CCL5, CCL11, CXCL12, and TGFß2 were measured in each AqH sample from control patients undergoing routine cataract surgery and patients with idiopathic uveitis, FHC, herpes-viral and Behçets uveitis (Table 1) . Levels of IL-6, IL-8, IFN
, and CCL2 were significantly increased in idiopathic uveitis AqH, compared with noninflammatory controls (Fig. 2 , Table 1 ). Both TGFß2 and CXCL12 were detectable in noninflammatory control AqH, but these levels were significantly reduced in idiopathic uveitis AqH. For IL-2, -12, and -13, there was a low level present in control AqH which decreased to be undetectable in the majority of idiopathic uveitis AqH. For all the other cytokines and chemokines tested, levels were either below detection level or did not significantly differ between idiopathic uveitis and control AqH. When comparing treated and untreated idiopathic uveitis AqH, the only significant differences were elevated IL-8 (untreated 248, treated 787 pg/mL; P < 0.05) and decreased TGFß2 (untreated 182, treated 27 pg/mL; P < 0.05) in the treated group.
|
|
Cluster Analysis of Cytokine Profiles in Idiopathic, FHC, Behçets and Herpes-Viral Uveitis, and Noninflammatory Control Aqueous Humor
Traditional univariate statistical analysis allows the determination of cytokines that are significantly altered between two groups of samples. However, to understand the complex relationships between cytokines and to predict which cytokines might allow discrimination of sample populations, some form of multivariate analysis is required. Cluster analysis was performed on the levels of cytokines and chemokines for all AqH, and showed that noninflammatory control and idiopathic uveitis AqH tended to cluster on different branches of the dendrogram (Fig. 3) . IL-6, IL-8 and CCL2, cytokines that were significantly elevated in uveitis AqH, clustered at one end of the dendrogram, with TGFß2 and CXCL12, which were both decreased, at the other end. We applied random forest analysis to the data. This resulted in a 100% classification accuracy between noninflammatory control and idiopathic uveitis AqH (Fig. 4A) . The importance of individual cytokines in classifying controls from idiopathic uveitis AqH was also calculated. Only IL-6, IL-8, CCL2, IL-13, IL-2, and TNF
were required for maximum classification accuracy (Fig. 4B) . Clustering was also observed for 2/4 herpes-viral, 4/5 FHC, and 3/4 Behçets uveitis AqH. The numbers of samples would have to be increased for each group to observe any significant clustering or to apply random forest analysis.
|
|
(r = 0.75, P < 0.0001) and between TGFß2 and CXCL12 (r = 0.87, P < 0.0001; Fig. 5 , Table 2 ). By contrast, the levels of TGFß2 (or CXCL12) inversely correlated with IL-6 (r = 0.74, P < 0.0001) and IL-8 (r = 0.53, P < 0.01).
|
|
, and the cellular infiltrate in idiopathic uveitis AqH (Fig. 5 , Table 3 ). The most significant were IL-8 with the total number of cells and IL-6 with the number of neutrophils. There was an inverse correlation between levels of TGFß2 and the number of leukocytes and neutrophils. CXCL12 also showed an inverse correlation with neutrophil numbers. These data show close associations between cytokines in uveitis AqH and with the cellular infiltrate present.
|
| Discussion |
|---|
|
|
|---|
Multiplexed bead-based immunoassays provide a powerful technique to identify cytokines and chemokines that are likely to play significant roles in the pathology of uveitis. However, although the use of multiplexed bead-based immunoassays is now well established,27 28 29 30 there are a number of issues that merit discussion. With any new technology, it is important to establish that the results are consistent with previous technologies. A direct comparison of ELISA and bead assays revealed that the patterns of cytokines detected were the same in each case.34 The authors detected variation between suppliers, but this applied to both bead-based and ELISA assays. Therefore, the bead-based assays do not appear to suffer from any greater variation than that found with more conventional analysis. As with conventional analysis, it is also necessary to ensure that measurements are reproducible. We performed repeated analyses for IL-6, IL-8, and IFN
on a few samples, and, although the levels were not always identical, there was an excellent correlation (r = 0.91, P < 0.0001; data not shown). The data from in vitrostimulated T cells also shows the low level of error for triplicate samples (Fig. 1B) . We do not consider this to be any greater a problem than for conventional assays, and multiplexed assays have the advantage of reduced variability between different cytokines, because they are all measured in the same sample.
Once the reliability of multiplexed bead immunoassays is established, the next challenge is to analyze the large amount of data generated. We performed standard statistical analysis by using Dunns multiple comparison, because these tests correct for multiple comparisons. It is important to appreciate that if sufficient comparisons are performed, then, by chance, a significant difference is likely to be detected. Only by using statistical tests that correct for multiple comparisons can this issue be circumvented, and, even under these conditions, considerable caution must be exercised when drawing conclusions. In addition to traditional statistical analysis, we performed both cluster and random forest analysis. Cluster analysis allows a visual profile to be established, giving an indication of the relationship between samples and cytokines. Although there is no quantitative statistical power to these profiles, they are often very useful. To classify the samples into control noninflammatory and uveitis AqH, we used random forest analysis. This method has the advantage of determining which molecules are required for maximal classification of the sample groups, therefore indicating which cytokines might play key roles in pathogenesis of the disease.
CD4+ T lymphocytes are able to induce disease in experimental models and are found at high frequencies in uveitis AqH.10 The cytokine responses of CD4+ T lymphocytes can be polarized to a TH1 (IFN
) or TH2 (IL-4) phenotype. Although the majority of experimental uveitis models induce a strong TH1 response that correlates with susceptibility to disease,35 TH2 cells are also able to induce experimental uveitis.36 Studies of human uveitis have favored a TH1 dominated response; however, this issue remains unresolved.37 Our results definitively show that uveitis AqH is dominated by TH1 cytokines. We could not detect any of the TH2 cytokines IL-4, -5, or -13, whereas IFN
was significantly elevated in idiopathic uveitis and levels correlated with the numbers of all infiltrating leukocyte populations. The highest levels of IL-13 were actually found in noninflammatory control AqH and decreased in uveitis.
IL-12 is a heterodimeric complex made up of p35 and p40 subunits, able to polarize toward TH1-type T cell responses. We were unable to detect significant elevations in the level of IL-12, instead, the very low levels found in noninflammatory controls decreased in uveitis AqH. The levels of uveitis AqH and vitreous IL-12 reported in the literature are inconsistent. Studies by el Shabrawi et al.8 showed elevated IL-12 in the AqH and vitreous from active uveitis patients in the range 72 to 293 pg/mL. However, other studies of vitreous38 and AqH9 have failed to find these elevations. It is unclear why these discrepancies exist. Our assay detects the active p70 subunit of IL-12, whereas other assays may detect the p40 subunit that can be found in excess. It is also possible that IL-12 has not reached a peak in our samples, all of which were of very recent onset.
The etiology of uveitis is unknown but is assumed to be (auto)immune in nature. However, it is clear that a number of proinflammatory cytokines are elevated in the local ocular environment during episodes of the disease. We confirmed that high levels of IL-6, IL-8, CCL2, and IFN
can be found in idiopathic uveitis AqH. Additionally, there were correlations between the levels of IL-6, IL-8, and IFN
, suggesting that these cytokines are interdependent or co-regulated. IL-6 has been reported to antagonize the anti-inflammatory actions of TGFß2,39 and we have recently reported that this cytokine protects uveitis AqH lymphocytes from apoptosis.40 In contrast to IL-6, the anti-inflammatory cytokine TGFß2, measured here in its active form, decreased as the proinflammatory cytokines increased, consistent with the loss of immune regulation seen in experimental uveitis.21 39 The same pattern was seen for the constitutive chemokine CXCL12, which has been shown to be downregulated by inflammatory cytokines.41 42
By applying cluster and random forest analysis to the cytokine and chemokine AqH data, we were able to classify idiopathic uveitis AqH from noninflammatory AqH with 100% efficiency. This classification only required the measurement of IL-6, IL-8, CCL2, IL-13, IL-2, and TNF
. This suggests that these cytokines are key molecules in the pathogenesis of idiopathic uveitis. Interestingly, IL-13, IL-2, and TNF
decreased in uveitis and were only found at very low levels in noninflammatory controls. The biological significance of this is unclear. It may reflect a true decrease in production or increased consumption of cytokine during inflammation. It is important to realize that AqH cytokines will only provide a reflection of the processes occurring within the ocular tissues, where local levels of some cytokines may be much higher. We performed all studies from patients with active disease and did not consider it ethical to take samples as inflammation resolves. Although we did not have the opportunity in this study, it would be very interesting to study the longitudinal profile of cytokines in patients with persistent disease.
As well as idiopathic uveitis, we analyzed the patterns of cytokines in AqH from FHC, herpes-viral uveitis, and Behçets disease. Due to the small numbers of samples, it was not possible to draw firm conclusions or to perform random forest analysis. However, there were clear differences for some cytokines, and a degree of clustering was observed, suggesting that future studies with larger numbers of samples might reveal distinct cytokine/chemokine profiles in these groups of samples.
IL-10 is considered to be an immunoregulatory cytokine able to limit pathologic damage and is produced by some regulatory T cell subsets.43 The relative concentration of IL-10 to IL-6 has been suggested as a useful marker to distinguish intraocular lymphoma from uveitis, although this issue is still controversial.25 38 Our own data, in conjunction with published data on IL-10 in infectious uveitis,12 clearly shows that in uveitis IL-10 is only present in combination with high levels of IL-6. Hence, our ratios of IL-10:IL-6 were 0.0030.21 (median 0.03), in contrast to the ratios reported for intraocular lymphoma, which are in excess of 1.0. In a study by Ongkosuwito et al., 12 IL-10 was found in the AqH from 10/17 herpes-viral uveitis patients with a range of 29 to 3927 pg/mL but only 3/51 control samples that included 21 idiopathic uveitis cases, as well as cases of Toxoplasma gondii uveitis. Therefore, it seems likely that the majority of infectious uveitis will induce significant IL-10 production. The presence of IL-10 could reflect a host antiviral response, an attempt to regulate inflammatory damage and/or be an attempt by the pathogen to escape immune intervention. The
-Herpes virus family carry many homologues of cytokines, chemokines, and receptors to deviate the host immune response, and these include IL-10.44 It is possible that the
-Herpes-viruses (VZV, HSV-1 and -2), which do not carry this array of homologues, might instead induce a deviant host immune response, possibly involving IL-10. During the course of experimental autoimmune uveitis, IL-10 appears late in the disease45 and has been proposed to be involved in the resolution of inflammation. Recently, peripheral blood lymphocytes from patients receiving anti-TNF
therapy were shown to secrete increased levels of IL-10,46 though the significance of these results is currently unclear. IL-10 secreting lymphocytes have also been identified in both idiopathic and FHC uveitis.47 We detected IL-10 in only 3/22 idiopathic uveitis AqH, which may reflect the immunoregulatory role proposed for this molecule. The absence of any clustering of these samples with the herpes-viral group suggests that these idiopathic cases do not represent undiagnosed herpes-viral uveitis.
Multiplex bead immunoassay analysis is a powerful tool with the ability to measure many different cytokines in a single sample, requiring only a 50-µL volume. This allows the quantification of each cytokine, and the examination of their relationship to each other and to the clinical characteristics of the disease. The application of multiplex bead immunoassays to uveitis AqH has revealed a close association between cytokines and the cellular infiltrate, as well as distinctive cytokine profiles in certain clinical uveitis entities. These studies will add to our understanding of the pathogenesis of uveitis and will be useful in determining the effects of the current range of biological therapies.
| Footnotes |
|---|
Submitted for publication April 8, 2005; revised June 29, 2005; accepted September 7, 2005.
Disclosure: S.J. Curnow, None; F. Falciani, None; O.M. Durrani, None; C.M.G. Cheung, None; E.J. Ross, None; K. Wloka, None; S. Rauz, None; G.R. Wallace, None; M. Salmon, None; P.I. Murray, None.
The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked "advertisement" in accordance with 18 U.S.C.
1734 solely to indicate this fact.
Corresponding author: S. John Curnow, Institute of Biomedical Research, Division of Immunity and Infection, Medical School, The University of Birmingham, Birmingham B15 2TT, UK; s.j.curnow{at}bham.ac.uk.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
G. Shi, A. Maminishkis, T. Banzon, S. Jalickee, R. Li, J. Hammer, and S. S. Miller Control of Chemokine Gradients by the Retinal Pigment Epithelium Invest. Ophthalmol. Vis. Sci., October 1, 2008; 49(10): 4620 - 4630. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. Simon, A. K. O. Denniston, P. J. Tomlins, G. R. Wallace, S. Rauz, M. Salmon, P. I. Murray, and S. J. Curnow Soluble gp130, an Antagonist of IL-6 Transsignaling, Is Elevated in Uveitis Aqueous Humor Invest. Ophthalmol. Vis. Sci., September 1, 2008; 49(9): 3988 - 3991. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. D. Shao, R. Suresh, V. Vakil, R. H. Gomer, and D. Pilling Pivotal Advance: Th-1 cytokines inhibit, and Th-2 cytokines promote fibrocyte differentiation J. Leukoc. Biol., June 1, 2008; 83(6): 1323 - 1333. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Banerjee, V. Savant, R. A. H. Scott, S. J. Curnow, G. R. Wallace, and P. I. Murray Multiplex Bead Analysis of Vitreous Humor of Patients with Vitreoretinal Disorders Invest. Ophthalmol. Vis. Sci., May 1, 2007; 48(5): 2203 - 2207. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Pladzyk, A. B. M. Reddy, U. C. S. Yadav, R. Tammali, K. V. Ramana, and S. K. Srivastava Inhibition of Aldose Reductase Prevents Lipopolysaccharide-Induced Inflammatory Response in Human Lens Epithelial Cells Invest. Ophthalmol. Vis. Sci., December 1, 2006; 47(12): 5395 - 5403. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. G.-J. Ooi, G. Galatowicz, V. L. Calder, and S. L. Lightman Cytokines and Chemokines in Uveitis - Is there a Correlation with Clinical Phenotype? Clin. Med. Res., December 1, 2006; 4(4): 294 - 309. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. Takase, Y. Futagami, T. Yoshida, K. Kamoi, S. Sugita, Y. Imai, and M. Mochizuki Cytokine profile in aqueous humor and sera of patients with infectious or noninfectious uveitis. Invest. Ophthalmol. Vis. Sci., April 1, 2006; 47(4): 1557 - 1561. [Abstract] [Full Text] [PDF] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||