PDFClick here to download the MIXX Awards 2014 Judging Handbook.
Password to open: mixxjudges2014
 
DEADLINE: Monday, August 11 @ 11:59pm ET
Questions? Contact Gina Imperato, IAB MIXX Awards Executive Producer at
[email protected] or 917-696-6370.
 
 
 

Thank you for volunteering your time and expertise to support the IAB MIXX Awards. For ten years, the IAB MIXX Awards judging panel has played a crucial role in ensuring only the best work gets recognized. We appreciate your time and effort.

Below is everything you need to know to complete your review including instructions for how to use the online system as well as criteria for reviewing and scoring the entries. We strongly recommend you print this document and entry category descriptions as you’ll want to refer to them frequently during your review.

If you have any questions or issues, please don’t hesitate to contact Gina Imperato at [email protected] or 917-696-6370.

I. About IAB MIXX Awards

The IAB MIXX Awards are about great work that drives results. Therefore, try not to get distracted by the amazing visuals in creative samples /case study videos or the impressive metrics in written brief. Instead, look for a combination of a great idea, with beautiful visuals, that is executed brilliantly and has made real impact against its stated objective. Think about what makes this work stand out and why it would be a great example to educate the industry about what works in interactive and help the marketplace understand where the industry is headed in the future.

II. Before You Start – The Basics

  1. Confidentiality agreement:
    All the information contained in the entries is confidential. Many entries contain behind-the-scenes details that shed light on competitive secrets and advantages. Per the IAB confidentiality agreement you signed, you cannot use or share any of the information contained in the entries. PLEASE NOTE: You will not be able to access the online system until your signed confidentiality agreement is received.
     
  2. The review mechanics and timeline:
    The entries you and your fellow jurors review in advance of the day-long sequestering in NYC on Wednesday, August 13, will allow us to spend more time discussing the merits of the shortlist work rather than reading entry forms. This part of the judging is done entirely online. As long as you have a computer and internet connection, your review can be done from anywhere at anytime between Friday, August 1 and Monday, August 11.
    PLEASE NOTE: Since many of the case study videos require flash, reviews cannot be done on an iPad.

    Each judge will be reviewing only the higher-order categories – Strategies & Objectives and the Special Innovation – Can’t be Contained! This will be 65 entries across 13 sub-categories. If you have time, we’d also like you to give a quick review (no scoring required) of approximately 18 gold winners in the Tools, Tactics & Platforms categories.

    It is absolutely critical that you complete the review of the 12 Strategies & Objectives and the one Special Innovation categories no later than Monday, August 11 at 11:59pm ET to ensure IAB can prepare the judging day materials for the sequestering in NYC on Wednesday, August 13.
     
  3. Some tips:
    Conflict of interest. If your company has been shortlisted (congratulations!) or you have some association with any shortlisted work, DO NOT REVIEW ANY OF THE ENTRIES IN THE CATEGORY (yours or the others). We've eliminated the obvious ones but if we missed one, please use the "Report a Conflict" link. Please do a quick scan of the list so you don't end up reviewing part of the category, coming across a conflict and doing a bunch of work you didn't have to.

    Problem viewing videos, broken links or the entry form. We have combed through the links and everything is working properly. However, issues are bound to come up. If you have a problem accessing creative or any portion of the entry form, please use the "Submit a Problem" link. We strongly recommend that you start each entry by reviewing the creative portion of the entry form, as this typically gives you a good overview of the entry that helps when you read the written brief.

    Personal Scorecard. You'll notice there is a notes field in the online review system for each entry. We encourage you to use this feature to jot down any thoughts, questions or feedback you would like to remember when we meet in person on the August 13th. Anything you record in the notes, along with your scores, will be stored in the system and will be included in your personal "scorecard." You'll receive a printed copy of this scorecard on the judging day so you'll know exactly how you scored each entry and see any notes you made. It will also include how your scores compare to the group’s scores overall.

    A word about Special Innovation – Can’t be Contained! category. Special Innovation Category - Can't Be Contained! will work a little differently than last year and is different from the other categories. Like the other categories, you will review 5 nominees and score them. However, on judging day, instead of deciding who takes home gold, silver and bronze, you’ll decide which three of the five are invited to present their work live on stage at the IAB MIXX Awards and compete for the People’s Choice Award. What hasn’t changed? Per advice from you, the judges, we want to recognize work that is so cutting-edge and future focused, it points to where digital marketing is headed. As the drivers of the industry, you have the best insight into what illustrates that. Therefore, we wanted to give you the opportunity to review the work entered here and be the ones who determine how to narrow it down into the best of the best.

    Review of Tools, Tactics & Platforms categories. As many of you know, we made some changes to the types and number of categories this judging panel reviews. Due to these changes, this jury is ONLY doing a comprehensive review and scoring of the five finalists in each of the 12 Strategies & Objectives categories plus the Special Innovation - Can’t be Contained! category. However, we did want to give you the opportunity to see the top scoring entry in each of the Tools, Tactics & Platforms categories in case there is work you might want the group to consider during the Best in Show discussions. For this reason, you’ll see one entry (potentially two if there was a tie) for each of the 18 Tools, Tactics & Platforms categories. You will not score these entries. You will simply review the creative and give the written brief a quick scan.

III. Getting Started – What to keep in mind/look for while evaluating entries

  1. What category is the entry submitted in? Be sure to review the entry in the context of the category in which it has been submitted. (See entry category document for entry category descriptions.)
     
  2. Each entry is scored on four components (Creative samples, Objective/Goal, Idea and Execution, and Results) — and each accounts for 25% of the overall score. The only exception is the Search Marketing category, which doesn’t require creative samples so the creative component is omitted for these entries.
     
  3. Review the creative samples before reading the written brief. All creative samples and/or case study videos will be accessible via URL or FTP provided by the submitting firm. Some entries will require a username and/or password. If required, this information will be included in the entry form that you are reviewing.
     
  4. Scoring guidelines. While each of the components is weighted equally, each one must be rated on a scale of 0-10. Please keep in mind the following questions while scoring.
    •  
    • Creative samples: Did the video and/or creative samples give you a good sense of the work and the interactive experience? Did you get a feel for the consumer experience?
       
    • Objective/Goal: Do you understand what the brand was trying to accomplish? Did the brief outline the strategic business or marketing challenge the work was created to address? (e.g. Improve overall brand perception? Create a unique consumer engagement? Increase market share? Increase brand awareness? Drive $x unit sales? Etc.)
       
    • Idea and Execution: Do you know what the “big idea” behind this entry is and understand the plan developed to meet the brand’s goals stated in the previous section? Did the entrant outline the key insight(s) that informed the work and strategic plan for accomplishing the goals outlined in the objectives/goals section? (e.g. What was the inspiration behind the work? What interactive tools, tactics, platforms were use to execute the vision? Why were they chosen? How were those tools used to make the work unique and noteworthy? Etc.)
       
    • Results: Did the entrant clearly articulate how they and the client knew the work was successful in meeting the objectives and goals outlined in that section? Did the entrant outline how the work was evaluated, and how they/ the client knew the work accomplished its goals? (e.g. What KPIs or metrics show that the work met or exceeded the brand’s expectations? What effect did the work have on the brand’s business? Beyond clicks and shares, what else was important in evaluating the impact of the work? Etc.)

      Below is further guidance for the points on the rating scale.
      • 0 = doesn’t meet requirements
      • 1 - 4 = below average
      • 5 = average
      • 6 - 9 = above average
      • 10 = hit a home run

IV. How to Review and Score Entries

  1. Log into MIXX Awards Screening website at:
    http://www.iab.net/mixxsubmissions/users/login

    If you are logging in for the first time, click the “Start Judging” button on the right hand side. All entries in the final running will appear.

    We suggest you score all the entries in the same category at the same time in order to grade entries comparatively.
    1. Your personal user name is: see “IAB MIXX Awards Judging Panel Instructions” email
    2. Your password is: see “IAB MIXX Awards Judging Panel Online Instructions” email
       
  2. Conflict of interest:
    1. If you have a conflict of interest (your company is a finalist or you have some association with any entry you are asked to review), click the "conflict of interest" link. A dialog box will appear.
    2. Enter a brief description explaining the conflict of interest and click "Submit and Skip This Category". If you were viewing an entry, you will be brought back to the Submission list.
       
  3. Scoring an entry:
    1. Click on the entry you wish to review. The full entry will appear.
    2. From there you can review each section of the entry. The URLs for the supporting creative are at the top of each entry and will open in a popup window when clicked. If you have trouble viewing them in the popup window, you can use the external link [->] icon to launch the URL in a new window. Some creative will require a username and password. This information is provided in the entry form.
    3. After you review each section and the creative, please record your scores (0-10) in the “Score” section at the bottom of each entry form.
    4. Enter any useful notes pertaining to your review in the “Scoring Notes” text box.
    5. Once you’ve designated a score for each of the four sections, click the “Save” button at the bottom.
    6. You will be brought to back to the Submission list.
       
  4. Problems with creative:
    1. If you have trouble accessing Creative URLs or viewing Written Case Studies, click the "report a problem" link, located to the right of each section. A dialog box will appear.
    2. Enter a description of the problem, and press "Submit Problem Report.”
       
  5. Logging out and taking a break:
    1. You can log out then continue judging at any time. To do this, click the “Logout” button at the top right hand corner of your screen. Your progress will be saved.
    2. To resume judging, simply log in again.
       
  6. Reviewing and editing scores:
    1. To view the Submission list at any time, click on the “review” link located in the right hand sidebar, next to the progress indicator.
    2. To edit your scores for any entry, click on the entry title.
    3. Adjust the scores for the entry as needed, then press “Save.” You will be returned to the Submission list.
       
  7. Finalizing your scores:
    1. When you have reviewed each of the finalists’ entries, reviewed your scores, and are certain that you are finished, you must click “Finalize Scores” in the right hand side bar (the Finalize Scores button will only appear when you have completed scoring all required entries). Your scores will be finalized and submitted to the central MIXX Awards database. After finalizing your scores, you will be locked out and will no longer be able to access the system.
    2. Your online review of the finalists is not considered complete until you finalize your scores.


If you have questions, please contact Gina Imperato, IAB MIXX Awards Executive Producer ASAP at [email protected] or 917-696-6370.