| 000 | 03705nla2a2200529 4500 | ||
|---|---|---|---|
| 001 | 659478 | ||
| 005 | 20231030041630.0 | ||
| 035 | _a(RuTPU)RU\TPU\network\28050 | ||
| 035 | _aRU\TPU\network\27958 | ||
| 090 | _a659478 | ||
| 100 | _a20190221d2018 k y0engy50 ba | ||
| 101 | 0 | _aeng | |
| 105 | _ay z 100zy | ||
| 135 | _avrcn ---uucaa | ||
| 181 | 0 | _ai | |
| 182 | 0 | _ab | |
| 200 | 1 |
_aBuilding analytical platform with Big Data solutions for log files of PanDA infrastructure _fA. A. Alekseev [et al.] |
|
| 203 |
_aText _celectronic |
||
| 225 | 1 | _aMathematical simulation and data processing | |
| 300 | _aTitle screen | ||
| 320 | _a[References: 6 tit.] | ||
| 330 | _aThe paper describes the implementation of a high-performance system for the processing and analysis of log files for the PanDA infrastructure of the ATLAS experiment at the Large Hadron Collider (LHC), responsible for the workload management of order of 2M daily jobs across the Worldwide LHC Computing Grid. The solution is based on the ELK technology stack, which includes several components: Filebeat, Logstash, ElasticSearch (ES), and Kibana. Filebeat is used to collect data from logs. Logstash processes data and export to Elasticsearch. ES are responsible for centralized data storage. Accumulated data in ES can be viewed using a special software Kibana. These components were integrated with the PanDA infrastructure and replaced previous log processing systems for increased scalability and usability. The authors will describe all the components and their configuration tuning for the current tasks, the scale of the actual system and give several real-life examples of how this centralized log processing and storage service is used to showcase the advantages for daily operations. | ||
| 461 | 1 |
_0(RuTPU)RU\TPU\network\3526 _tJournal of Physics: Conference Series |
|
| 463 | 1 |
_0(RuTPU)RU\TPU\network\28043 _tVol. 1015 : Information Technologies in Business and Industry (ITBI2018) _oInternational Conference, January 17-20, 2018, Tomsk, Russian Federation _o[proceedings] _fNational Research Tomsk Polytechnic University (TPU) _v[032003, 6 p.] _d2018 |
|
| 610 | 1 | _aэлектронный ресурс | |
| 610 | 1 | _aтруды учёных ТПУ | |
| 610 | 1 | _aплатформы | |
| 610 | 1 | _aBig Data | |
| 610 | 1 | _aлог-файлы | |
| 610 | 1 | _aинфраструктура | |
| 610 | 1 | _aвысокопроизводительные системы | |
| 610 | 1 | _aпрограммное обеспечение | |
| 610 | 1 | _aданные | |
| 610 | 1 | _aобработка | |
| 610 | 1 | _aхранение | |
| 701 | 1 |
_aAlekseev _bA. A. _cspecialist in the field of automatic control _cEngineer of Tomsk Polytechnic University, Postgraduate _f1988- _gAleksandr Aleksandrovich _2stltpush _3(RuTPU)RU\TPU\pers\37959 |
|
| 701 | 1 |
_aBarreiro Megino _bF. G. |
|
| 701 | 1 |
_aKlimentov _bA. A. |
|
| 701 | 1 |
_aKorchuganova _bT. A. |
|
| 701 | 1 |
_aMaendo _bT. |
|
| 701 | 1 |
_aPadolski _bS. V. |
|
| 712 | 0 | 2 |
_aНациональный исследовательский Томский политехнический университет _bИнженерная школа неразрушающего контроля и безопасности _bЦентр промышленной томографии _bНаучно-производственная лаборатория "Бетатронная томография крупногабаритных объектов" _h7983 _2stltpush _3(RuTPU)RU\TPU\col\23717 |
| 801 | 1 |
_aRU _b63413507 _c20150101 _gRCR |
|
| 801 | 2 |
_aRU _b63413507 _c20190228 _gRCR |
|
| 856 | 4 | _uhttp://dx.doi.org/10.1088/1742-6596/1015/3/032003 | |
| 856 | 4 | _uhttp://earchive.tpu.ru/handle/11683/52921 | |
| 942 | _cCF | ||