Sarakuna yana ba da masana'anta, kwalabe na abin sha, da masu samar da abinci a duk faɗin duniya tare da injuna guda ɗaya da cikakkun layin samarwa. Kowace rana, miliyoyin kwalabe na gilashi, gwangwani, da kwantenan PET suna tafiya ta layin Krones. Layukan samarwa wasu tsare-tsare ne masu yawa tare da yuwuwar kurakurai waɗanda za su iya dakatar da layin da rage yawan samarwa. Krones yana so ya gano gazawar da wuri-wuri (wani lokaci ma kafin ya faru) kuma ya sanar da masu aikin layin samarwa don haɓaka aminci da fitarwa. To ta yaya ake gano gazawa? Krones yana ba da layukan su tare da na'urori masu auna firikwensin don tattara bayanai, waɗanda za a iya kimanta su ta hanyar ƙa'idodi. Krones, a matsayin mai kera layin, da kuma ma'aikacin layin suna da damar ƙirƙirar ka'idodin kulawa don injuna. Don haka, kwalaben abin sha da sauran masu aiki za su iya ayyana nasu gefen kuskure don layin. A da, Krones ya yi amfani da tsarin da ya danganci tsarin bayanai na lokaci. Babban ƙalubalen shine cewa wannan tsarin yana da wahalar cirewa sannan kuma tambayoyin suna wakiltar yanayin injina a halin yanzu amma ba sauyin yanayi ba.

Wannan sakon yana nuna yadda Krones ya gina mafita mai gudana don saka idanu akan layin su, bisa ga Kinesis na Amazon da kuma Sabis na Gudanar da Amazon don Apache Flink. Waɗannan cikakkun ayyukan da aka sarrafa suna rage rikitaccen ginin aikace-aikacen yawo tare da Apache Flink. Sabis ɗin Gudanarwa don Apache Flink yana sarrafa tushen abubuwan Apache Flink waɗanda ke ba da yanayin aikace-aikacen ɗorewa, awo, rajistan ayyukan, da ƙari, kuma Kinesis yana ba ku damar sarrafa bayanan yawo cikin farashi mai inganci a kowane sikeli. Idan kuna son farawa da aikace-aikacen Apache Flink naku, bincika Bayanan ajiya na GitHub don samfurori ta amfani da Java, Python, ko SQL APIs na Flink.

Bayani na mafita

Sa ido kan layin Krones wani bangare ne na Krones Shop Floor Guidance tsarin. Yana ba da tallafi a cikin ƙungiya, fifiko, gudanarwa, da takaddun duk ayyukan da ke cikin kamfanin. Yana ba su damar sanar da ma'aikaci idan an dakatar da injin ko ana buƙatar kayan, ba tare da la'akari da inda mai aiki ke cikin layi ba. An riga an gina ingantattun ƙa'idodin sa ido akan yanayin amma kuma ana iya siffanta mai amfani ta hanyar mai amfani. Misali, idan wani wurin bayanan da ake sa ido ya sabawa kofa, ana iya samun saƙon rubutu ko faɗakar da odar kiyayewa akan layi.

An gina tsarin kula da yanayin yanayi da tsarin kimanta mulki akan AWS, ta amfani da ayyukan nazarin AWS. Zane mai zuwa yana kwatanta tsarin gine-gine.

Kusan kowane aikace-aikacen yawo da bayanai ya ƙunshi nau'i biyar: tushen bayanai, shigar da rafi, ajiyar rafi, sarrafa rafi, da wuri ɗaya ko fiye. A cikin sassan da ke gaba, mun zurfafa zurfi cikin kowane Layer da kuma yadda tsarin sa ido kan layi, wanda Krones ya gina, ke aiki daki-daki.

Tushen bayanai

Ana tattara bayanan ta hanyar sabis da ke gudana akan na'urar gefen da ke karanta ƙa'idodi da yawa kamar Siemens S7 ko OPC/UA. An riga an tsara ɗanyen bayanai don ƙirƙirar haɗin kan tsarin JSON, wanda ke sauƙaƙa aiwatarwa daga baya a injin ƙa'ida. Nauyin samfurin kuɗin da aka canza zuwa JSON na iya zama kamar haka:

{
  "version": 1,
  "timestamp": 1234,
  "equipmentId": "84068f2f-3f39-4b9c-a995-d2a84d878689",
  "tag": "water_temperature",
  "value": 13.45,
  "quality": "Ok",
  "meta": {      
    "sequenceNumber": 123,
    "flags": ["Fst", "Lst", "Wmk", "Syn", "Ats"],
    "createdAt": 12345690,
    "sourceId": "filling_machine"
  }
}

Ciwon rafi

Abubuwan da aka bayar na AWS IoT Greengrass buɗaɗɗen tushen Intanet ne na Abubuwa (IoT) lokacin gudu da sabis na girgije. Wannan yana ba ku damar yin aiki akan bayanai a cikin gida da tarawa da tace bayanan na'urar. AWS IoT Greengrass yana ba da abubuwan da aka riga aka gina waɗanda za'a iya tura su zuwa gefe. Maganin layin samarwa yana amfani da sashin mai sarrafa rafi, wanda zai iya sarrafa bayanai da canja wurin shi zuwa wuraren AWS kamar Binciken AWS IoT, Sabis na Sauƙi na Amazon (Amazon S3), da Kinesis. Mai sarrafa rafi yana tattara bayanai da tara bayanai, sannan aika shi zuwa rafin bayanan Kinesis.

Ma'ajiyar rafi

Ayyukan ma'ajiyar rafi shine adana saƙonni ta hanya mara kyau da sanya shi don amfani ga ɗaya ko fiye aikace-aikacen mabukaci. Don cimma wannan akan AWS, fasaha na yau da kullun shine Kinesis da Gudanar da Gudanarwar Amazon don Apache Kafka (Amazon MSK). Don adana bayanan firikwensin mu daga layin samarwa, Krones zaɓi Kinesis. Kinesis sabis ne na bayanan yawo mara sabar wanda ke aiki a kowane sikeli tare da ƙarancin latency. Shards a cikin rafin bayanai na Kinesis wani tsari ne na musamman da aka gano na bayanan bayanan, inda rafi ya ƙunshi shards ɗaya ko fiye. Kowane shard yana da 2 MB/s na ƙarfin karantawa da ƙarfin rubuta 1 MB/s (tare da max 1,000 records/s). Don guje wa bugun waɗannan iyakoki, ya kamata a rarraba bayanai tsakanin shards daidai gwargwado. Kowane rikodin da aka aika zuwa Kinesis yana da maɓallin ɓangarori, wanda ake amfani da shi don tara bayanai cikin shard. Don haka, kuna son samun maɓallan ɓangarori masu yawa don rarraba kaya daidai gwargwado. Manajan rafi da ke gudana akan AWS IoT Greengrass yana goyan bayan ayyukan maɓalli na ɓangarori, wanda ke nufin cewa duk bayanan sun ƙare a cikin sharar bazuwar kuma ana rarraba kaya daidai gwargwado. Rashin lahani na ayyukan maɓalli na ɓangaren bazuwar shine cewa ba a adana bayanai cikin tsari cikin Kinesis. Mun yi bayanin yadda za a warware wannan a cikin sashe na gaba, inda muke magana game da alamar ruwa.

Alamar ruwa

A watermark wata hanya ce da ake amfani da ita don ganowa da auna ci gaban lokacin aukuwa a cikin rafin bayanai. Lokacin taron shine tambarin lokaci daga lokacin da aka ƙirƙiri taron a tushen. Alamar ruwa tana nuna ci gaban aikace-aikacen sarrafa rafi, don haka duk abubuwan da suka faru tare da tambarin lokaci ɗaya ko daidai ana ɗaukar su azaman sarrafa su. Wannan bayanin yana da mahimmanci don Flink don ciyar da lokacin taron da kuma haifar da ƙididdiga masu dacewa, kamar kimantawar taga. Ana iya daidaita lag da aka yarda tsakanin lokacin taron da alamar ruwa don tantance tsawon lokacin da za a jira bayanan marigayi kafin yin la'akari da kammala taga da haɓaka alamar ruwa.

Krones yana da tsarin ko'ina a duniya, kuma ana buƙatar kula da masu zuwa marigayi saboda asarar haɗin gwiwa ko wasu matsalolin hanyar sadarwa. Sun fara ne ta hanyar sa ido kan masu shigowa marigayi da saita tsohowar Flink zuwa ga iyakar ƙimar da suka gani a cikin wannan awo. Sun fuskanci al'amura tare da aiki tare na lokaci daga na'urori na gefen, wanda ke jagorantar su zuwa ga ingantaccen hanyar alamar ruwa. Sun gina alamar ruwa ta duniya ga duk masu aikawa kuma sun yi amfani da mafi ƙarancin ƙima azaman alamar ruwa. Ana adana tambura a cikin HashMap don duk abubuwan da ke shigowa. Lokacin da alamun ruwa ke fitowa lokaci-lokaci, ana amfani da mafi ƙarancin ƙimar wannan HashMap. Don guje wa tsayawar alamomin ruwa ta hanyar bacewar bayanai, sun saita wani idleTimeOut siga, wanda ke watsi da tambarin lokutan da suka girmi wani kofa. Wannan yana ƙara latency amma yana ba da daidaiton bayanai mai ƙarfi.

public class BucketWatermarkGenerator implements WatermarkGenerator<DataPointEvent> {
private HashMap <String, WatermarkAndTimestamp> lastTimestamps;
private Long idleTimeOut;
private long maxOutOfOrderness;
}

sarrafa rafi

Bayan an tattara bayanan daga na'urori masu auna firikwensin kuma an shigar da su cikin Kinesis, yana buƙatar ingin doka ya kimanta shi. Doka a cikin wannan tsarin tana wakiltar yanayin ma'auni ɗaya (kamar zazzabi) ko tarin ma'auni. Don fassara ma'auni, ana amfani da ma'aunin bayanai fiye da ɗaya, wanda ƙididdigewa ne. A cikin wannan sashe, mun zurfafa zurfin cikin maɓalli da yanayin watsa shirye-shirye a cikin Apache Flink da yadda ake amfani da su don gina injin mulkin Krones.

Sarrafa rafi da tsarin yanayin watsa shirye-shirye

A cikin Apache Flink, jihar yana nufin ikon tsarin don adanawa da sarrafa bayanai akai-akai a cikin lokaci da aiki, yana ba da damar sarrafa bayanan yawo tare da goyan bayan ƙididdiga na jihohi.

The tsarin watsa shirye-shiryen jihar yana ba da damar rarraba jihar zuwa duk daidaitattun lokuta na ma'aikaci. Saboda haka, duk ma'aikata suna da jiha ɗaya kuma ana iya sarrafa bayanai ta amfani da wannan jiha. Ana iya shigar da wannan bayanan karatu-kawai ta amfani da rafi mai sarrafawa. Rafi mai sarrafawa shine rafin bayanai na yau da kullun, amma yawanci tare da ƙananan ƙimar bayanai. Wannan tsarin yana ba ku damar sabunta jihar a hankali akan duk masu aiki, yana bawa mai amfani damar canza yanayi da halayen aikace-aikacen ba tare da buƙatar sake aiki ba. Fiye da daidai, rarraba jihar ana yin ta ta hanyar amfani da rafi mai sarrafawa. Ta ƙara sabon rikodin a cikin rafi mai sarrafawa, duk masu aiki suna karɓar wannan sabuntawa kuma suna amfani da sabon jihar don sarrafa sabbin saƙonni.

Wannan yana bawa masu amfani da aikace-aikacen Krones damar shigar da sabbin dokoki cikin aikace-aikacen Flink ba tare da sake kunna shi ba. Wannan yana guje wa raguwa kuma yana ba da babban ƙwarewar mai amfani yayin da canje-canje ke faruwa a ainihin lokacin. Ƙa'ida ta ƙunshi yanayi don gano karkacewar tsari. Wani lokaci, bayanan injin ba su da sauƙin fassara kamar yadda ake iya kallon kallon farko. Idan na'urar firikwensin zafin jiki yana aika ƙima mai girma, wannan na iya nuna kuskure, amma kuma shine tasirin tsarin kulawa mai gudana. Yana da mahimmanci a sanya ma'auni a cikin mahallin kuma tace wasu dabi'u. Ana samun wannan ta hanyar ra'ayi da ake kira tarawa.

Rukunin ma'auni

Rukunin bayanai da ma'auni yana ba ku damar ayyana mahimmancin bayanan mai shigowa da samar da ingantaccen sakamako. Bari mu bi misalin da ke cikin wannan adadi.

A mataki na 1, mun ayyana ƙungiyoyin yanayi guda biyu. Rukuni na 1 yana tattara yanayin injin kuma wane samfurin ke tafiya cikin layi. Rukuni na 2 yana amfani da ƙimar zafin jiki da na'urori masu auna matsa lamba. Ƙungiyar sharadi na iya samun jihohi daban-daban dangane da ƙimar da take karɓa. A cikin wannan misali, ƙungiyar 1 tana karɓar bayanan da injin ke gudana, kuma an zaɓi kwalban lita ɗaya azaman samfurin; wannan ya baiwa wannan kungiya jihar ACTIVE. Rukuni na 2 yana da ma'auni don zafin jiki da matsa lamba; duka ma'auni suna sama da ma'auni na sama da mintuna 5. Wannan yana haifar da rukunin 2 kasancewa cikin a WARNING jihar Wannan yana nufin ƙungiyar 1 ta ba da rahoton cewa komai yana da kyau kuma ƙungiyar 2 ba ta yi ba. A mataki na 2, ana ƙara ma'auni zuwa ƙungiyoyi. Ana buƙatar wannan a wasu yanayi, saboda ƙungiyoyi na iya ba da rahoton bayanai masu karo da juna. A cikin wannan yanayin, ƙungiyar 1 ta ba da rahoto ACTIVE da rahoton rukuni 2 WARNING, don haka ba a bayyana wa tsarin menene yanayin layin ba. Bayan an ƙara ma'aunin nauyi, ana iya sanya jihohin, kamar yadda aka nuna a mataki na 3. A ƙarshe, an zaɓi jiha mafi girma a matsayin wadda ta yi nasara, kamar yadda aka nuna a mataki na 4.

Bayan an tantance ƙa'idodin kuma an ayyana yanayin injin na ƙarshe, za a ƙara sarrafa sakamakon. Matakin da aka ɗauka ya dogara da tsarin tsarin mulki; wannan na iya zama sanarwa ga ma'aikacin layi don dawo da kayan aiki, yin wasu gyare-gyare, ko kawai sabuntawar gani akan dashboard. Wannan ɓangaren tsarin, wanda ke kimanta ma'auni da ƙa'idodi kuma yana ɗaukar ayyuka bisa sakamakon, ana kiransa a injin mulki.

Ƙimar injin mulki

Ta hanyar barin masu amfani su gina nasu dokokin, injin mulki na iya samun adadin dokoki masu yawa waɗanda yake buƙatar kimantawa, kuma wasu dokoki na iya amfani da bayanan firikwensin iri ɗaya kamar sauran dokoki. Flink tsari ne da aka rarraba wanda ke da ma'auni sosai a kwance. Don rarraba rafin bayanai zuwa ayyuka da yawa, zaka iya amfani da keyBy() hanya. Wannan yana ba ku damar raba rafin bayanai ta hanya mai ma'ana da aika sassan bayanan zuwa ga manajojin ayyuka daban-daban. Ana yin wannan sau da yawa ta hanyar zabar maɓalli na sabani don ku sami kaya mai rarraba daidai gwargwado. A wannan yanayin, Krones ya ƙara a ruleId zuwa wurin bayanai kuma ya yi amfani da shi azaman maɓalli. In ba haka ba, bayanan da ake buƙata ana sarrafa su ta wani aiki. Za a iya amfani da rafin bayanan da aka maɓalli a cikin duk ƙa'idodi kamar maɓalli na yau da kullun.

destinations

Lokacin da doka ta canza yanayinta, ana aika bayanin zuwa rafin Kinesis sannan ta hanyar Abincin AmazonBridge ga masu amfani. Ɗaya daga cikin masu amfani ya ƙirƙira sanarwa daga taron da aka watsa zuwa layin samarwa kuma yana faɗakar da ma'aikata suyi aiki. Don samun damar bincika canjin yanayin ƙa'idar, wani sabis yana rubuta bayanai zuwa ga wani DynamoDB na Amazon tebur don saurin shiga kuma TTL yana cikin wurin don saukar da tarihin dogon lokaci zuwa Amazon S3 don ƙarin rahoto.

Kammalawa

A cikin wannan sakon, mun nuna muku yadda Krones ya gina tsarin sa ido kan layin samar da lokaci akan AWS. Sabis ɗin Gudanarwa don Apache Flink ya ba ƙungiyar Krones damar farawa da sauri ta hanyar mai da hankali kan haɓaka aikace-aikacen maimakon abubuwan more rayuwa. Ƙarfin lokaci na Flink ya ba Krones damar rage lokacin na'ura da kashi 10% kuma ya ƙara haɓaka har zuwa 5%.

Idan kuna son gina naku aikace-aikacen yawo, duba samfuran da ke akwai akan Bayanan ajiya na GitHub. Idan kuna son tsawaita aikace-aikacen Flink ɗinku tare da masu haɗin al'ada, duba Samar da Sauƙi don Gina Masu Haɗi tare da Apache Flink: Gabatar da Async Sink. Ana samun Async Sink a cikin Apache Flink version 1.15.1 da kuma daga baya.

Game da Authors

Florian Mair Babban Masanin Gine-ginen Magani ne kuma ƙwararriyar watsa bayanai a AWS. Shi masanin fasaha ne wanda ke taimaka wa abokan ciniki a Turai su yi nasara da haɓaka ta hanyar warware matsalolin kasuwanci ta amfani da sabis na AWS Cloud. Bayan yin aiki a matsayin Masanin Gine-ginen Magani, Florian ɗan hawan dutse ne, kuma ya haura wasu manyan tsaunuka a duk faɗin Turai.

Emil Dietl Babban Jagoran Fasaha ne a Krones ƙwararre a injiniyan bayanai, tare da maɓalli mai mahimmanci a Apache Flink da microservices. Ayyukansa sau da yawa sun haɗa da haɓakawa da kiyaye software mai mahimmancin manufa. Bayan rayuwarsa ta sana'a, yana matuƙar mutunta ciyar da lokaci mai kyau tare da danginsa.

Simon Peyer shine Magani Architect a AWS da ke Switzerland. Shi mai aikatawa ne kuma yana da sha'awar haɗa fasaha da mutane masu amfani da sabis na AWS Cloud. Babban abin da ya fi mayar da hankali a gare shi shine watsa bayanai da sarrafa kansa. Ban da aiki, Simon yana jin daɗin iyalinsa, a waje, da tafiya a cikin duwatsu.

• Ƙunshin Ƙarfafa SEO & Rarraba PR. Samun Girma a Yau.
• PlatoData.Network Tsaye Generative Ai. Karfafa Kanka. Shiga Nan.
• PlatoAiStream. Web3 Mai hankali. Ilmi Ya Faru. Shiga Nan.
• PlatoESG. Carbon, CleanTech, Kuzari, Muhalli, Solar, Gudanar da Sharar gida. Shiga Nan.
• PlatoHealth. Biotech da Clinical Trials Intelligence. Shiga Nan.
• Source: https://aws.amazon.com/blogs/big-data/krones-real-time-production-line-monitoring-with-amazon-managed-service-for-apache-flink/