You know and love our Must-Read IT Blogs lists, but now, say hello to the nonprofit side.
If you don’t believe that everything is bigger in Texas, then take a look at the new stadium for the Dallas Cowboys.
The $1 billion-plus, 3.2-million-square-foot facility in Arlington, Texas, is the largest domed stadium in the NFL. Two steel arches that soar 320 feet above the playing field support a retractable roof that is the longest single-span roof structure in the world. The new home for the storied football franchise is an architectural marvel that features bleeding-edge technology to give fans the best experience possible, including the world’s largest high-definition video board hanging from the roof above the 50-yard line.
Two video screens facing the sidelines are 70 feet tall and 60 yards in length, spanning the field from one 20-yard line to the other. Two large end-zone video boards are also attached to the roof, giving every fan access to live game footage as well as instant replays and pre- and post-game shows that feature interviews with players and coaches.
“What makes these video boards unique is there’s no bad seat in the house. You get a great view of the game whether you’re looking on the field or at the video boards,” says Scott Purcel, the Cowboys’ broadcasting director.
Since February, Cowboys CIO Pete Walsh and his team of 11 IT staffers have raced to install the IT infrastructure in the new stadium, which includes a new data center with 127 Hewlett-Packard blade servers and a new 100 terabyte storage area network (SAN) using HP’s StorageWorks 8100 Enterprise Virtual Array systems. The team built a high-speed communications network with Cisco Systems equipment that includes Wi-Fi, IP phones and an IP television system that will broadcast content to 2,900 flat-screen Sony televisions throughout the stadium, showing live game footage, advertising and menus at the concession stands. The team also installed 185 IP security cameras to safeguard the facility.
The new data center supports not only the team’s operations but team owner Jerry Jones’ 30-plus companies in more than 90 locations. That’s a total of 400 employees, including real estate developments, two MRI centers, oil and gas companies and 35 pro shops that sell team apparel and other merchandise.
“We support the football club, the cheerleaders, the scouting and medical staff, as well as the concession stands and the merchandise stores,” Walsh says. “But it’s not just football, so you can understand the logistics and challenges we have trying to keep everything running.”
Providing 24x7 uptime is critical for all the businesses, but for the football team, it’s particularly important on game day. To improve reliability and prevent downtime, the IT department equipped each server with two power supplies, each rack of servers with duplicate copper and fiber connections, and each row of servers with backup switches, says Bill Haggard, the team’s director of enterprise infrastructure. The staff also installed two power distribution units in each row of servers; and if there’s a power outage, a diesel generator is available to keep IT operations running.
Dallas Cowboys New Stadium’s field is 50 feet below ground level. The entire Statue of Liberty and its base could fit inside the stadium with the roof closed.
“We designed redundancy into the data center,” Haggard says. “We can’t lose connectivity or have downtime because 2,900 screens could go blank and could result in no advertising, no menus and no concession sales.”
To ensure there’s plenty of bandwidth, the team laid 250 miles of fiber-optic cable throughout the stadium and built 69 wiring closets. The largest closets, such as the one near the ticket office and Hall of Fame area, feature eight Cisco switches and four 10Gbps Fibre Channel connections to the data center. The smaller closets feature three Cisco switches with two 10Gbps Fibre Channel connections. The team doesn’t yet need all the bandwidth or have enough data to fill its 100TB SAN, but it purchased more than it needed to meet future requirements, Walsh says.
“We wanted to future-proof it,” he says. “Our big business partners like HP, Cisco, Sony and several others gave us access to their road maps and their vision of where technology is going, and that helped us make our decisions. We picked technology and infrastructure that will take us to the next 15 to 20 years.”
To improve reliability, the IT staff is building a second 100TB SAN at the team’s headquarters at Valley Ranch in Irving, Texas, so it can mirror data with the stadium SAN. Eventually, the IT staff will use disk-based backups, sending daily backups from one data center to another. But for now, the team will rely on tape backup using HP Data Protector backup software, Walsh says.
The IT department implemented new technologies aimed at helping employees work smarter while boosting sales and improving customer service. For example, the team’s new point-of-sale system, running on Microsoft Dynamics AX software, provides executives with real-time sales information. So during a game, if the weather turns cold and sales of fleece pullovers in one end of the stadium skyrocket, employees can evaluate inventory and replenish the supply at a particular store before they are sold out, Walsh says.
The stadium’s 665 point-of-sale terminals will also let fans pay with their credit and debit cards for the first time, which will drive sales, Walsh says. Previously, at Texas Stadium, every sale was a cash transaction.
“We were losing a significant amount of revenue,” he recalls. “[Customers] were standing in line. They couldn’t see the game, and they weren’t buying sodas or hot dogs.”
Dallas Cowboys New Stadium, the working title for the new facility, becomes the third home for the team. The Cowboys, an expansion franchise for the NFL in 1960, spent its first decade playing in the Cotton Bowl.
The team moved to Texas Stadium in 1971, a decade in which the team won its first two Super Bowls. Texas Stadium, which cost $35 million, is a 900,000-square-foot facility with a seating capacity of 64,675. In contrast, the new stadium cost more than $1 billion, takes up 3.2 million square feet, seats 80,000 and can expand to 100,000. Old Texas Stadium is scheduled to be demolished in 2010.
The IT staff also is installing 700 wireless access points. Ticket-takers are equipped with wireless handheld devices to scan tickets, while staffers at bars and clubs will have wireless handheld terminals to take orders, Haggard says.
The Cowboys, a perennial playoff contender and five-time Super Bowl champion, are well-known for relying on technology to give themselves a competitive edge. With custom software and a database full of archived game footage, the team’s coaching and scouting staff can analyze the tendencies of opponents, as well as the performance of their own players, to help develop game plans, Walsh says.
“Scouts who want to see every pass that [quarterback] Tony Romo completed over 20 yards can do a database query, and within seconds it will bring up the video of every completion over 20 yards,” Walsh says. “We can also look at the tendencies of teams inside the 20-yard line or their tendencies on third down. And with that, we can make better game decisions and build a better game plan.”
Like the new stadium, the Cowboys’ new locker room is equipped with projectors and motorized projection screens attached to the ceiling, so coaches and players can review game footage, Walsh says.
While implementing the technology infrastructure, the IT staff spent months this spring migrating Cowboys employees from Texas Stadium to the new stadium. The team is standardized on Microsoft, so they moved applications such as Active Directory and Exchange Server. The IT staff took a phased approach, moving one group of employees at a time, such as food and beverage employees and the ticket office, Haggard says.
During the migration, the IT staff lugged new servers to Texas Stadium, used VMware virtual machines to take snapshots of applications and data, and then brought the servers back to the new stadium. The IT team then mirrored the data between the two locations, and once the equipment in the new stadium was synced with the data, the staff turned off the servers at the old stadium, Haggard says.
The IT department is taking advantage of server virtualization to better utilize its blade servers. The IT staff is using 212 VMware virtual machines to run the point-of-sale terminals in the concession stands. Each concession stand has its own virtual machine because the Radiant point-of-sale software requires that each concession uses its own Windows Server for processing, Haggard says. Another 30 virtual machines are used as file and printer servers for employees’ day-to-day operations, he adds. The team also bought 25 HP ProLiant DL380 rack servers to run the stadium’s video system and IP security cameras.
Overall, the Cowboys’ IT staff has logged some long hours to get the stadium ready, but they are looking forward to unveiling the technology this summer and for years to come. “The new stadium is a showcase for technology,” Haggard says. “All the technology that is going into the stadium is state of the art for any location, for any business, not just football.”