In today’s day & age, many services are being accessed by users through a client application on mobile. What’s the point of five 9s of server availability, if the productionized app is unreliable for users? This talk explains the importance, challenges and best practices of client-side reliability.

 
 

Outline/Structure of the Talk

Why Mobile and SRE (Site Reliability Engineering)?

  • Mobile is nonuniform and uncontrollable
  • SRE responsibilities differ in critical ways from infrastructure or server-side application reliability engineering
  • Focus on where your end users are (i.e. today many users access services through mobile applications)
  • Specific challenges of mobile
    • Monitoring
    • Release management
    • Incident management

Case studies

  • Doodle outage, etc.

Future of SRE for Mobile

  • Visibility into mobile application performance
  • Find and react to issues before the user does
  • Measure SLIs throughout the product stack (from client to service)

Learning Outcome

Client side is the most tangible experience of your service as fas as the user is concerned. So just measuring 9s of availability on the server side is not enough.,

Target Audience

Mobile Developers and DevOps Engineers who do not think about client side impact of their services.

schedule Submitted 3 months ago

  • 20 Mins
    Talk
    Beginner

    When I refresh my memory and consider my whole testing life, I again realize that life is full of lessons. Whenever I think about this, I come up with a saying by John W. Gardner: “Life is the art of drawing without an eraser.” We are all making mistakes and it is not possible to manage each single piece of processes perfectly. Still; what we can remember is, with additional brush strokes, those mistakes can evolve into nice colors inside the big picture. Besides, in the next run, we may take precautions against possible mistakes.

    In short, after the quick review of my career, I decided to collect the required soft skills for a tester. Being a tester requires lots of skills, but it is not holding all of them in an unnatural way. We are humans, no supermans.

     
  • Hiroyuki Ito/伊藤 宏幸
    keyboard_arrow_down

    Hiroyuki Ito/伊藤 宏幸 - Testable Infra: Cloud-native技術をフル活用した、「テスト」の諸問題の刷新的解決案

    45 Mins
    Talk
    Advanced

    皆さんは、システムのテストをする際に、次のような問題を経験されたことはないでしょうか?

    • テスト環境の準備の工数がバカにならない
    • テストデータを複数人・チームで共有していて、気軽にテストできない
    • 外部サービスと接続できず、必要なテストができない

    これらの問題は、開発者・QAなどのシステム関係者にとって、テストに対する非常に大きな技術的・心理的障壁となります。加えて、十分なテストを行えないことによる本番障害の多発にもつながります。

     

    私たちのチームは、これらの問題を解決すべく、Cloud-native技術を駆使して、2020年に「Testable Infra」という社内インフラを構築し、運用を開始しました。加えて、これを複数のシステム開発プロジェクトに導入し、実際に上述のテストの諸問題の改善という成果が出始めています。

     

    このセッションでは、「Testable Infra」のコンセプト、設計と技術の詳細、およびこれを活用したシステム開発の未来像についてお話しします。

    なおこのセッションは、2020年12月に中国で開催された「TOP100 Summit 2020」で発表した『Make testing easier technically and psychologically with Kubernetes - Innovation of testing distributed systems with "Testable Infra" -』を大幅に加筆修正し、日本語で発表します。

  • Masanori Kawarada (Mark Ward)
    keyboard_arrow_down

    Masanori Kawarada (Mark Ward) - 独立QAチーム1年戦記:スクラムの外からチームと組織の品質を創る道 / An Independent QA Team's 1 Year's War: Way to Create Quality of the Teams and the Organization from the Outside of Scrum

    45 Mins
    Talk
    Intermediate

    English follows:

    「Scrum@Scale(S@S)を取り入れた100名ほどの開発組織で、スクラムに入らない独立したQAチームが活躍している」と聞いたら、もしかしたら奇異な感じを受けるかもしれない。スクラムではチームであらゆることが完結することを理想とするため、当然テスター(QAエンジニア・テストエンジニア ・などなど)もスクラムチームに入るべき、と考える方がスクラム実践者にとっては自然だからである。あえて、その自然に逆らって、私たちQAチームは独自のビジョンを掲げた「ビジョナリー・QA(Visionary QA)」として、独立した立場から品質向上という成果を上げようと奮闘している。このトークではそんな私たちQAチームの1年戦記をありのままに扱いたい。

    開発プロセスの高速化が進み、多くの企業でアジャイル開発が取り入れられ、過去の当たり前が当たり前でなくなっている中で、QA界隈ではどうやって価値を提供するか頭を悩ませている。あくまでもテストにこだわる者もいれば、スクラムマスター・プロダクトオーナーの視野を得るべく資格を目指す者もいる。2009年に翻訳出版されたリサとジャネットによる『実践アジャイルテスト(Agile Testing)』(翔泳社)は国内のテスターに広く読まれているが、最近版元品切になっていることもあり、手に入りにくくなっている。

    さて、テスターは異質な存在のひとつとして見なされている。ご存知の通り、スクラムガイドにはテストやQA活動をどのように行うべきか、言及されていない。3つのロールに「テスター」の文字は無い。よって、テスターのあり方はそれぞれの組織で模索するしかなく、特にテスターをスクラムチームに含めるべきか否かという議論は継続的に行われている。先にもあげたように、スクラム実践者にとってはテスターがスクラムチームに入ることは自然であると感じられ、実際そのようにしている組織が多いが、それぞれにメリット・デメリットがあることから、あらゆる組織で通じる答えは今のところ無さそうだ(テスターとして仕事をしてきたメンバーがスクラムチームの開発者の一員としてどれだけクロスファンクショナルに動けるか、という点が特にネックなように思える)。

    このトークは、独立した立場で動くことを選んだQAチームの話だ。スクラムチームにテスターを迎え入れねばならないと思っている方にはそうでない可能性を示す一方で、やはりスクラムチームに開発者としてテスターを加えるべきだと結論づけるオーディエンスもいらっしゃるかもしれない。スクラムチームとテスターの一筋縄ではいかない関係を、1年間の実例をもとに、一緒に考えようではないか。

    "In a 100-strong software development organization which runs Scrum@Scale, an independent QA team works well." ––it may sound strange to you. Ideally, in Scrum, everything should be done in one scrum team, so it is natural for Scrum practitioners that testers (QA engineers, test engineers, etc.) should join a scrum team. Daring to go an unnatural way, our QA team struggles to achieve quality improvement results from an independent standpoint as "Visionary QA" with its vision. I want to treat our QA team's year-long battle story as it is in this talk.

    Development processes are getting faster and faster. Many companies are incorporating agile development. The commonplace of the past is no longer the norm. In this fast-changing age, the QA industry is struggling to figure out how to deliver value.  Some testers are more concerned with testing, while others aim for the certification to learn a Scrum Master/Product Owner's perspective. The excellent book, "Agile Testing" by Lisa Crispin and Janet Gregory (Addison-Wesley), which was translated in Japan in 2009 by the domestic publisher Shoeisha, has been widely read by testers in our country. Recently, however, it isn't easy to get due to out of print.

    Testers tend to be seen as one of the heterogeneous entities. As you know, there is no mention in the Scrum Guide on how testing and QA activities should be done. There is no "Tester" in the three roles of Scrum. Therefore, each organization has no choice but to explore the nature of testers. In particular, there is an ongoing debate on whether or not testers should be included in Scrum. As mentioned earlier, it seems natural for Scrum practitioners to have testers join Scrum, and indeed many organizations are doing so. Still, since each has its advantages and disadvantages, it seems that we don't have an answer that works for all organizations at the moment. One of the problematic points appears to be how well testers can work cross-functionally as a "Developer" in Scrum.

    With this session, which is about a QA team that chose to work independently, some attendees who feel testers should join a Scrum team may get a chance to notice the other possibility, and others may conclude that Scrum teams should still include testers. Let's take a look at the tricky relationship between Scrum and testers with the help of a year's worth of examples.

  • Mesut Durukal
    keyboard_arrow_down

    Mesut Durukal - 4 Pillars of Successful Agile Test Automation

    20 Mins
    Talk
    Beginner

    Motivation:

    After executing a project to test a cloud-based microservices platform, we experienced a lot of challenges in both technical and social manners and tried to develop solutions to cope with them. Finally, I have wrapped them up to make a list of golden rules to successfully manage a test project. The purpose of this talk is to give insights about how a test automation project is managed.  

     

    Problems:

    • Technically, if not managed properly, automated testing will lead to extra costs and could even be less effective than manual testing.
    • From another aspect, at some point, we had too many complaints in our retrospective meetings about the heavy deployment activities and redundant executions. We had lots of flaky tests, execution lists full of not clear test definitions. Everyone was sick and tired of maintenance issues and the team was not happy. We took actions to improve motivation in the team. 

     

    Solutions:

    • Being truly agile: Adapt new solutions quickly. 
    • Manage the progress: Be aware of what is going on by setting KPIs to track & monitor with tools like CloudWatch, Grafana
    • Technical part: Automation principles for the sake of robustness. Solutions to reduce flaky tests & analysis effort & costs: Code demos on Java API polling libraries, Selenium usage and others.
      • How we solved flaky tests: Adaptations into the code and observing the effect of each application. A few examples are polling mechanisms (safe wait methods) and test design techniques 
      • For maintenance efforts: We keep test history in the pipelines and develop helper methods to improve quality. 
      • Improve coverage from different aspects to reduce escaped bugs.
    • Team spirit! 

     

    Results & Conclusion

    After application of the proposals, we observed that waste is eliminated by prioritization and removing duplicated or dispensable work. After all, we believe that this submission has interesting content which can make great attention. Instead of theoretical claims, we discuss faced challenges and applied solutions. We analyze effects of solutions with before-after situations, graphs and evidence. Instead of what to do, we go over how to do it. 

     
  • 45 Mins
    Talk
    Beginner

    Psychological safety has been identified as the topmost feature of a successful and innovative organization. At the same time, we need to learn from failure and prevent recurrence of mistakes. These two practices seem to contradict each other, but is there a way to achieve them both?

help