Leadership in AI: Is Your Leadership Fit for Data Science?

Conventional people leadership has worked well for decades. People leadership focuses on managing and guiding individuals, teams, and organizations. This can involve setting goals and objectives, providing support and guidance, and creating a positive work environment that fosters collaboration and innovation.

In contrast, leadership in data science involves using data and analytics to drive decision-making and business strategy. This may involve working with data scientists and other professionals to collect, analyze, and interpret data, and using this information to inform business decisions and strategies.

Both people leadership and leadership in data science require similar skills, such as the ability to communicate effectively, think strategically, and solve problems. However, the specific focus and responsibilities of these roles can be quite different. People leaders may be more focused on managing people and teams, while leaders in data science may be more focused on using data and analytics to drive business strategy.

When people leaders grew up in the same business as workers. Leaders had an in-depth understanding of the duties, difficulties, and preferences of the workers. However, in the field of data science, non-technical people managers who transitioned into data science leadership often lack a technical understanding of concepts, methods, and the function of data scientists. This results in communication gaps and mismatched expectations leading to under-performance and lack of work satisfaction, and churn.

Photo by Possessed Photography on Unsplash

The expectations for people leaders in the field of AI will depend on the specific organization and role. In general, people leaders in AI may be expected to —

• Understand the latest developments and trends in AI technology, and use this knowledge to inform strategy and decision-making.
• Develop and implement strategies to foster collaboration and innovation among teams working on AI projects.
• Manage and guide individuals and teams working on AI projects, setting goals and objectives and providing support and guidance as needed.
• Communicate effectively with stakeholders, including explaining technical concepts and the potential implications of AI technology to a non-technical audience.
• Collaborate with other leaders and teams within the organization to align AI projects with broader business goals and strategies.

Overall, the expectations for people leaders in AI will involve combining technical expertise in AI with strong people management skills to drive the successful development and implementation of AI projects.

People managers transitioning into data science leaders often creates gaps in technical knowledge. These gaps generally end up causing friction within the team, adding inefficiencies and a general feeling of dissatisfaction. As the growth of AI in organizations picks up, the skillset gap of the conventional leader and the responsibilities diverge even further. This results in the teams being highly vulnerable to the issues discussed above. One way the leadership tries to fill the knowledge gap is by choosing one of the many certification programs synonymous with “AI for leadership” or “Leadership in DataScience”. IMHO, these courses are not helpful on their own.

Photo by Katie Moum on Unsplash

Theories of Leadership

Before we go any further, let us go through some of the leadership theories that might be relevant to the discussion -

Great man theory

A popular concept in the 19th century, this theory asserts that you can’t develop leadership abilities — you either have them, or you don’t. The great man theory of leadership states that great leaders are born with all the right personality traits, such as intelligence, courage, confidence, intuition, and charm.

Trait theory

The trait theory of leadership states that certain natural qualities tend to create good leaders. Some leaders may be good listeners or communicators, but not every listener or communicator makes a good leader.

Behavioral theory

The behavioral theory of leadership focuses on how a person’s environment, not just natural abilities, condition a person to be a leader. It asserts that a person will be more likely to act or lead in a certain style as a result of environmental responses to behavior. Probably, the oversimplification of behavioral theory is that anyone can be a leader if they behave the way other leaders do.

Transactional or management theory

The transactional theory of leadership, also called “management theory,” states that leadership is a system of rewards and penalties. It views effective leadership as results-focused and hierarchical. Transactional leaders prioritize order and structure over creativity by rewarding someone who meets a goal and penalizing someone who doesn’t.

Transformational or relationship theory

The transformational theory of leadership states that effective leadership is the result of a positive relationship between leaders and team members. Transformational leaders motivate and inspire through their enthusiasm and passion. They are a model for their teams, emphasizing a collaborative work environment, diplomatic communication skills, and efficient delegation.

Situational theory

The situational theory of leadership does not relate to a certain type of leader or claim that any one style is best. Instead, it asserts that the best kind of leader is one who can adapt their style based on the situation. They may respond to a situation by commanding, coaching, persuading, participating, delegating, or however they think is necessary. Situational leaders are defined by their flexibility.

Photo by Nick Fewings on Unsplash

The theory that accommodates Data Science

Coming back to conventional leadership in DataScience, the leadership theories that may have a chance to find a fit are a transformational theory, Behavioral theory, or rarely Trait theory. Data science has grown exponentially in the last few decades. The data scientists are supposed to have multiple skill sets that overlap with other specialties like data engineering, programming, scientific thinking/topic familiarity/expertise, and even the ability to explain the reasoning and results to non-data science-heavy audiences. Furthermore, data science teams closely work with engineering teams, subject matter experts, and often non-scientific and scientific leadership.

I want to emphasize that the skillset (technical or otherwise) expected from the teams that work on data science products is wide in breadth. Such tall asks demand appropriately skilled immediate leadership (Data Science Directors, Sr. Data Science Directors). This minimizes the aforementioned gap reducing friction and increasing productivity.

A team member who has grown into the role by being in a similar role of working in a team and has not lost the mojo of his/her technical skills is a great fit for immediate leadership of technical teams. Building teams has a risk, rather than asking a non-technical leader to learn technical lingo by passing a few certificate programs, businesses should take a risk to grow the soft skills of leadership from the technical teams. It will likely result in a high-performing team.

credits: https://unsplash.com/@jasongoodman_youxventures

Leadership style that suites Data Science

1. Coach (motivational)
2. Visionary (progress-focused and inspirational)
3. Servant (humble and protective)
4. Autocratic (authoritarian and result-focused)
5. Laissez-faire or hands-off (autocratic and delegatory)
6. Democratic (supportive and innovative)
7. Pacesetter (helpful and motivational)
8. Transformational (challenging and communicative)
9. Transactional (performance-focused)
10. Bureaucratic (hierarchical and duty-focused)

A data science team’s internal candidate is likely to be

1. servant (the leader must be humble)
2. visionary (the leader who is inspirational to the teammates)
3. often democratic (no one knows the best answer, the team works together to agree on what should work the best)
4. pacesetter (the leader who can roll up their sleeves when needed)
5. transformational (the leader who can speak the same language and share the joy and pain of being a data scientist, and the team trusts him/her to speak on their behalf of them)
6. sometimes transactional (the leader who knows how to judge technical team members when the time comes and has the ability to focus on the team’s output)

A combination of these styles is generally highly appreciated by high-performing teams. High-performing teams do like to innovate with consensus, be humble about their abilities, and or be mentoring and motivational to the junior staff. With the right mentoring of soft skills, technical members of the team can transition into leaders who are less likely to cause friction within the team and more likely to follow the right technical practices. As an added bonus, they are likely to be more realistic about the progress, expectations, obstacles, and workarounds that might save businesses precious time and money.

People managers, a technical leader is your friend!

Unfortunately, time in time, non-technical people managers in data science end up suffocating the technical leaders by asking technical leaders to be the facilitator of communication between themselves and the team. Sometimes, this is followed by them attempting to broadcast it as their own knowledge to senior leadership. Lack of technical understanding of non-technical people managers leads to oversimplification of the project idea, often leading to unrealistic expectations from senior leadership. I urge non-technical people managers to avoid it at all costs.

For context, the job of the data scientist is never truly done. The projects that data scientists are tasked with are seldom short-term business commitments. Allow enough time for them to understand the asks. Allow them to communicate with designated members from other parts of the organization. Most importantly, leadership should allow them to understand the bigger picture of how end-user is affected.

Credits: https://unsplash.com/@ashkfor121

People managers who are unfamiliar with the technical aspects should identify technical leaders from the team. Ideally, technical leaders will be those who have technical experience, familiarity with the business, and the ability to communicate. People managers should delegate the responsibility of technical decisions to the technical leaders. People managers should utilize technical leaders as the facilitators of technical communication within and outside the team. A good facilitator of technical communications can often lead to better outcomes.

Final thoughts: Embrace the Grind!

People managers who aspire to be data science leaders have 2 options -

1. Learn, Learn, and Learn — until you are technically strong enough that the skill gap between team and leader is acceptably low.
2. Delegate, Delegate, and Delegate — Use technical leaders to your advantage until #1 above.

Some of the points in this article come from my personal experiences and some from my tech acquaintances.

People management is a necessary skill that technical leaders often lack. However, data science leadership is a whole different ball of wax. It takes a lot to learn a lot of things that are constantly evolving. I must confess that there are many days when I feel like an imposter. Upskilling in data science needs time and patience. Take your time and grow with the team!

Embrace the grind!

If this article helped you, Please share some love by keeping the like button pressed.

Please become a member and subscribe for more

Credits: https://unsplash.com/@brett_jordan